Creating multiple workers in the same process

I’m thinking of creating multiple worker processes using a for loop to listen and poll from the same task-queue. However, all the workers created have a default ID that is based on the client identity/pod name. Based on the logs, multiple workers were started but upon checking the temporal web UI, it only shows 1 active worker.

Does this create multiple worker entities correctly and is this the right practice?


for i := 0; i < numWorkers; i++ {
		w := worker.New(client, task-queue, workerOptions{})
		err := w.Start()
		if err != nil {
			return err
		}
	}

Why are you creating multiple workers for the same task queue in the same process? This is not ever needed.

Would having multiple workers for the same task queue in the same process allow the workers to process the more tasks from the same task queue?

No, it doesn’t bring any advantage.

Why though?

Because a single worker can saturate the process CPU/memory.

I have a related question. We have a requirement to execute activities (from different workflows) in parallel. In this case, is the recommendation to create separate task queues for each workflow and have one worker per task queue? (Instead of having a single task queue for all workflows, and multiple workers read from the same task queue)

I would start from a single task queue for everything and move to multiple task queues only if you want to break your processes into multiple binaries (services) or have specific rate-limiting requirements for a subset of activities.

If I have a single task queue for all workflows, how do I achieve parallel execution of activities with a single worker? By registering the worker for all activities?

Yes, you register all the activities and workflows with the same worker.

1 Like

Understood! Thank you for your quick response.

@maxim The Temporal.io “How It Works” Landing Page recommends that applications have at least two workers, but here you’re saying that a Task Queue only needs one worker. What is the distinction between the landing page and your advice? How it Works | Temporal Technologies

It doesn’t make sense to have multiple workers for the same task queue in the same process. “How it works” talks about multiple worker processes for availability.