My question
I’d like to better understand what happens if a workflow tries to run an activity but the dedicated worker for that activity has already hit maxConcurrentActivityTaskExecutions. My intuition was that the activity would be queued to be handled as soon as concurrentActivityTaskExecutions drops below the max and that activity’s startToCloseTimeout wouldn’t begin until it begins execution. Is this intuition correct? If not, can someone explain the actual behavior in that scenario.
Background info
I have some activities with global max concurrency limitations. To facilitate this, I’ve setup a dedicated worker and unique task-queue for each group of activities where max-concurrency needs to be enforced. This allows me to set a global maximum concurrency for each activity group via maxConcurrentActivityTaskExecutions parameter of their dedicated worker.
Hi,
Yes, your intuition is correct.
When the worker reaches maxConcurrentActivityTaskExecutions it will stop polling the server for more activity tasks until any running activities conclude (meaning your activity function returns or throws).
As a general strategy this approach works as long as your limit is low enough for a single worker to handle all of the concurrent tasks.
If your activities are CPU intensive you could use a thread pool like piscina to leverage more CPUs for processing.
Thanks @bergundy, the confirmation is helpful.
Appreciate the tip about piscana too, do have one activity w/concurrency max of 1500. It’s just a bunch of fetch req (low CPU intensive) and doesn’t seem to fail under heavy load. Is that too much strain to put on a single worker? Should I look to have piscana spawn threads to offload some of the burden?
If you see the worker can handle the load and the main thread isn’t blocked, keep everything in the main thread
1 Like