Workflow or Commands picked up by all workers

Community Support
,
1

My scenario:
I have multiple workers listening on the same task queue for high throughput.
All the workers need to do some setup and cleanup work from time to time.
My requirement:
the business client can send a workflow or a command, which can be picked up and executed by all workers, instead of only one of them.
Is there any solution?

2

Hi @whitecrow

I am afraid it is not possible, to route requests (workflow task / activity task) to specific workers you need different taskqueue names, one for each worker.

3

Just to add,

picked up and executed by all workers, instead of only one of them

Can you describe your use case please? Don’t think this requirement applies to Temporal. A workflow execution is not tied to a specific worker and can migrate workers (if for example some are down). A particular execution progress of your workflow code (a workflow task) can happen on only one worker at a time tho.

4

I filed the Notify all workers on a task queue issue to get this functionality added.

5

Thanks~, @maxim
Wish this new functionality can be released in a near future.

6

Maxim’s description is more accurate than mine.
What I try to achieve is a notification to all workers on a task queue. The notification may contains some parameters for the worker execution.

1 Like
7

@antonio.perez @tihomir @maxim
to have a workaround, is there an api which we can use to list all task queues under a namespace?
My idea is to setup a jvm-specific task queue in each worker process. If I can get the task queue list then I can send workflows as notifications to them.

1 Like
8

I would create a worker notification workflow. Each worker periodically would send a signal to that workflow. The workflow is going to maintain the list of active workers that sent a signal within configured timeout. Upon receiving notify signal it would go over the list and execute a notification activity in each of them.

9

This sounds a good solution. But it needs an assumption:
The “master” has no duplications, or we would encounter the same issue above: only one replication can pick up the heartbeat message.

10

Workflows are guaranteed to be unique for the given workflow ID.

11

Sorry, I don’t understand your last comment. What is master, replication and heartbeat have to do with sending a notification to each worker?

12

“master” I mean the service which needs to maintain active workers list and broadcast notifications. It may also have multiple instances, just as the workers. This is what I mean master duplications.
“heartbeat” I mean the notification workers send out to indicate its liveness.
So my concern is:
no matter the masters broadcast notifications, or workers report their liveness, there is always only one receiver getting the message.

13

Thanks for the explanation. In the design, I proposed there will be a single “master workflow execution.” As with any other workflow, it is not linked to a specific worker process. So there is no concept of master service in this design.

Messages are sent to the workflow through its ID, so there is a hard guarantee that only one workflow with this ID will receive all the messages.