Scaling Concurrent Execution of Activites and Workflows

Community Support
1

How does Workers execute workflows and activities concurrently?
Is it through Multithreading?

Does increasing maximum concurrent workflow execution and activity execution variables of Worker, result in increase in number of threads equally?

How to approach scaling worker for handling concurrent workflow and activity executions?

  • Increase the hardware resource of the system in which worker is running

  • Increase the maximum concurrent workflow execution and activity execution variables of Worker

  • Add a new worker to the pool.

What should be the order when scalling?

2

You want to saturate the worker first. Unless your activities are memory bound increasing the number of polling threads and concurrent activity and workflow tasks will increase CPU usage.

When the worker is CPU bound then you can either change the hardware to the one with more CPU capacity or add more workers to the pool. There is no generic recipe to which one is better for all the use cases.

1 Like
3

@maxim What will happen if there are many workflows started while only a small number of worker processes are available? I assume those scheduled workflows will stuck in a queue until they get picked up. Is that right? What problem it could introduce besides latency spikes, like, will the worker process get overloaded and crash? Thanks!

4

It will increase the latency of workflow execution. Workers are not going to be overloaded if they are configured correctly (setting a limit of parallelism activity and workflow tasks) as they ask for tasks only if there is a spare capacity.