I’m thinking of creating multiple worker processes using a for loop to listen and poll from the same task-queue. However, all the workers created have a default ID that is based on the client identity/pod name. Based on the logs, multiple workers were started but upon checking the temporal web UI, it only shows 1 active worker.
Does this create multiple worker entities correctly and is this the right practice?
for i := 0; i < numWorkers; i++ {
w := worker.New(client, task-queue, workerOptions{})
err := w.Start()
if err != nil {
return err
}
}
I have a related question. We have a requirement to execute activities (from different workflows) in parallel. In this case, is the recommendation to create separate task queues for each workflow and have one worker per task queue? (Instead of having a single task queue for all workflows, and multiple workers read from the same task queue)
I would start from a single task queue for everything and move to multiple task queues only if you want to break your processes into multiple binaries (services) or have specific rate-limiting requirements for a subset of activities.
If I have a single task queue for all workflows, how do I achieve parallel execution of activities with a single worker? By registering the worker for all activities?
@maxim The Temporal.io “How It Works” Landing Page recommends that applications have at least two workers, but here you’re saying that a Task Queue only needs one worker. What is the distinction between the landing page and your advice? How it Works | Temporal Technologies
It doesn’t make sense to have multiple workers for the same task queue in the same process. “How it works” talks about multiple worker processes for availability.
How about creating multiple workers for different task queues in the same process?
In my case I have many different activities defined, but not a huge workload at this moment.
So instead of having one worker with a huge list of activities registered to it, Can I have multiple workers with separate queues, in order to organize the activities into separate task queues.
I know that while one worker is blocking, the other workers can’t pull any tasks. but is there any limitations other than that? (I’m using typescript)
you’ll make both compete for CPU/memory if done on the same process. I’d have them in separate processes, or pods, or VMs, depending on how you’re running in production