My scenario:
I have multiple workers listening on the same task queue for high throughput.
All the workers need to do some setup and cleanup work from time to time.
My requirement:
the business client can send a workflow or a command, which can be picked up and executed by all workers, instead of only one of them.
Is there any solution?
Hi @whitecrow
I am afraid it is not possible, to route requests (workflow task / activity task) to specific workers you need different taskqueue names, one for each worker.
Just to add,
picked up and executed by all workers, instead of only one of them
Can you describe your use case please? Don’t think this requirement applies to Temporal. A workflow execution is not tied to a specific worker and can migrate workers (if for example some are down). A particular execution progress of your workflow code (a workflow task) can happen on only one worker at a time tho.
Maxim’s description is more accurate than mine.
What I try to achieve is a notification to all workers on a task queue. The notification may contains some parameters for the worker execution.
@antonio.perez @tihomir @maxim
to have a workaround, is there an api which we can use to list all task queues under a namespace?
My idea is to setup a jvm-specific task queue in each worker process. If I can get the task queue list then I can send workflows as notifications to them.
1 Like
I would create a worker notification workflow. Each worker periodically would send a signal to that workflow. The workflow is going to maintain the list of active workers that sent a signal within configured timeout. Upon receiving notify signal it would go over the list and execute a notification activity in each of them.
This sounds a good solution. But it needs an assumption:
The “master” has no duplications, or we would encounter the same issue above: only one replication can pick up the heartbeat message.
Workflows are guaranteed to be unique for the given workflow ID.
Sorry, I don’t understand your last comment. What is master, replication and heartbeat have to do with sending a notification to each worker?
“master” I mean the service which needs to maintain active workers list and broadcast notifications. It may also have multiple instances, just as the workers. This is what I mean master duplications.
“heartbeat” I mean the notification workers send out to indicate its liveness.
So my concern is:
no matter the masters broadcast notifications, or workers report their liveness, there is always only one receiver getting the message.
Thanks for the explanation. In the design, I proposed there will be a single “master workflow execution.” As with any other workflow, it is not linked to a specific worker process. So there is no concept of master service in this design.
Messages are sent to the workflow through its ID, so there is a hard guarantee that only one workflow with this ID will receive all the messages.