Multiple workers per microservice feedback

Community Support
1

Hello I have a general question about workers.

We have some services that start multiple workers, for different task-queues.

We typically start a couple of workers per micro-service:

  • 1 worker handles internal logic, so it registers activities and worklows on the “internal task-queue”, this logic is no meant to be consumed by other microservices.
  • 1 worker handles logic that could be consumed by other microservices, so this is registered on a “publicly known” task-queue
  • Another reason we like to sometimes split microservices into multiple worker/task-queue combinations is that we don’t need to worry about the Activity names since they only need to be unique within a task-queue (ofc this can be solved in other ways)

This idea of starting multiple workers per microservice, is it very bad ?

I found this comment in all of the go-sampels source code => “// The client and worker are heavyweight objects that should be created once per process.”

Currently we don’t really notice any problem with this, although our system typically dont experience very high load, we use temporal more to ensure corectness. But even though we dont experience any issues now we would like to follow good design. I do understand that 1 worker per service makes it easier to scale things that require more computing power, since you can just increase the amount of workers for that particular work.

Ty in advance!

2

This idea of starting multiple workers per microservice, is it very bad ?

Imho its not bad idea. Your approach would also allow you to have different release cycles (worker restarts) for internal vs external part of the service. Given that you use different task queues you can also scale the internal/external parts separately as needed.

“// The client and worker are heavyweight objects that should be created once per process.”

Think idea behind this statement are cases where you create client object inside activities rather than passing existing client (one you use to create your worker) when you register them with the worker.

From what I understand both your workers you start per task queue in same process are on same namespace, so they can reuse same client as well.