Estimating the right configuration values of the temporal services

Community Support Server Deployment
1

I am planning to deploy temporal in kubernetes cluster with AWS aurora RDS. I am having trouble estimating the right value of deployment configurations. Any approximations or ballpark numbers should be helpful.

Assume, I have to run 10k workflows per second with 10 activities each.
total tasks per second per workflow - workflow tasks (11) + activity tasks (10) = 21
total tasks per second = 210k

Questions

  1. How do I calculate the right number of shards required for my cluster to function properly?
  2. Any physical limitations around number of shards in SQL based datastores? Can it support 10k shards or more?
  3. As maxim mentioned in scaling-task-queue , we have to configure 100 partitions to support 10k tasks per second. For 210k tasks per second, 2100 partitions are required. Will the system work effectively with 2100 partitions (especially on mysql) ?
  4. what should be the service RPS for history, matching and frontend to avoid contextDeadlineExceeded and resource exhausted errors?
  5. To support such a scale, is it a good idea to increase max number of db connections by temporal services?

Thanks for helping

3

Hi, this sounds like very large use case. Are you sure about the 210k tasks per second estimation for it? Could you share more info on your use case?

Just double checking before we go into it further :slight_smile: