If I have 10 activities inside 1 workflow where I want to run some activity concurrently and some are not, all are synchronous and I put threadPoolExecutor(5) on worker.
Now if I trigger 10 workflows then each workflow has a thread of 5 ?
Certainly! Let’s break down the example:
I have a single workflow with 10 synchronous activities.
Set the ThreadPoolExecutor to 5 on workers.
If I trigger 10 workflows concurrently:
Each workflow operates independently and has its own ThreadPoolExecutor with 5 workers.
Therefore, I could potentially have 10 workflows running concurrently, with each workflow utilizing up to 5 threads for its activities. i.e., I cannot execute more than 5 activities concurrently for each workflow(each workflow means for same workflow I give 10 different types of inputs to excecute. Hence passing it in list/array of dictionary/object)
Is my understanding correct.
Which worker option are you talking about? There is activity_executor which only applies to activities and you should set with a thread pool executor, and it should have as least as many workers as max_concurrent_activities (newer SDK versions will warn if it doesn’t). This is just the thread pool used for activities on the task queue regardless of which workflow started them. There is also workflow_task_executor, but the default is almost always acceptable here and should not be changed.
I have 6 temporal worker instances
But i am taking about max_worker in threadpool
So in activity_executor=threadpool(5) I give and workflow contains 10 activities where first 3 i want to execute concurrent and other I don’t want to execute concurrently. This use case can I achieve by using await asyncio.gather(activity 1,2,3).
Is my understanding correct?
Note my all activities will be synchronous
Yes you are correct. The thread pooling of activity workers is unrelated to what the workflow says runs concurrently. Using asyncio tools to run things concurrently is the proper way. Thread pool size needs to be max concurrent activity size which needs to be based on how many the worker can handle for its resources.