Hi,
Trying to understand the worker thread creation from WorkerFactory. I have made some changes in sample HelloActivity sample to create workers on task queue in a loop fashion. (Line 52-57 in Screenshot)
From the logs, clearly able to see that pollers are getting created as below :
But still, when I view the pollers from Temporal Web UI, can see only 2 pollers on the same task queue name :
In the above image, I was expecting to see 5 pollers. Am I missing anything here? Looks like creating workers from different WorkerFactories on same task queue is not making any effect.
Also, are any metrics exposed, which can give an indication of no. of worker threads on a task queue or related things.
Please help.
Can someone plz shed some light on above query.
Are you sure you don’t have another worker running on this tq thats not created via your loop code?
Each worker created in your loop should have the same identity so you should see just one in web ui or via
tctl tq desc --tq HelloActivityTaskQueue
I think for worker creations you should start with single worker for a tq and then saturate it (see worker tuning in docs as well).
There is no need to create any pools or workers in a single Java process.
You do want to have multiple worker processes for availability and scalability.
@tihomir . I will check the result of tctl command once as you mentioned.
Also, when you say saturate, which parameters do you exactly mean? We have referred the worker tuning guide, and does that mean to increase poller count or execution size when we need to increase worker capacity?
Reason behind asking this is, we do not see a depletion in worker task slots available metrics, but still there is high ScheduleToStart latency even when persistence latency looks ok. Hence, we were trying the approach of creating multiple worker threads in a loop.
There is no practical reason to create more workers in the same process for the same task queue but rather scale them (via config options like provided and by deploying them on bigger hardware / give more memory/cpu for example). In case you need more workers create more worker processes.
Remember, workers execute your code, so they need to be deployed on production type of hardware imo.
For a single worker process it should be ok to create 1 WorkerFactory, and 1 worker per task queue.
So to put it in perspective of the sample you are working on, think of the “main” method it as one worker process. Instead of creating multiple workers for same tq inside a loop in one “main method/worker process” create multiple of these “main methods/worker” processes in separate classes in samples. Hope this helps.
1 Like
@tihomir . Yes, after doing a restart and confirming the same, I could see only 1 entry in Web UI as well as from tctl command.
We are performing load tests and figured out that the execution size slots are not being utilized(hardly 20-30 being consumed) even though the execution size is 200. Hence, we were trying to create multiple workers in the same process, and see if it helps in any way.
