Gotten a little familiar with the RingPop mechanism after digging into the SDK a bit. We are a bit concerned though on how we get nodes gossiping with each other when deploying each service of temporal as its own instance inside of ASGs.
The problem for us is that everything here is driven more by DNS than IP addresses - since IP addresses are dynamic. Due to some technical restrictions, we sort of have to start deploying this way vs. a kubernetes deployment in EKS.
Since IPs are non-static, how can we do this properly so that Temporal Cluster can see each of its nodes other? When dynamic auto-scaling kicks in, how would new nodes be incorporated into the fold (broadcasting)?
Is a service discovery system mandatory?
I am sure its something simple I just haven’t come across but it seems that there have been a few user related issues around this.
IP Addresses that were recycled end up being used in another cluster for example: Prevent incorrect service discovery with multiple Temporal clusters · Issue #1234 · temporalio/temporal · GitHub.
Again not an issue if driven by DNS, just need an example I guess if this is actually supported.