I need to send a signal to several thousand workflows from within a workflow. I see that SignalExternalWorkflow can be used for this. But, does signalling several thousand workflows from a single workflow like this contribute to the event history limit?
The docs only talk about hitting event history limit by having too many activities in a single workflow. But what about SignalExternalWorkflow?
If it does affect event history limit, do you think creating child workflows just to signal another workflow (no activities in the workflow) would affect performance negatively?
Yes, each time you send signal to external workflow it is recorded in history as ExternalWorkflowExecutionSignaled event.
On the receiving side, receiving signals is also recorded with the WorkflowExecutionSignaled event.
Both do count towards the 50K history limit.
do you think creating child workflows just to signal another workflow
You could partition the problem, let’s say you have 50K signals to send, could start 10 child workflows in parallel, that handle 5K each.
Typically you don’t want your workflow history to get super large, as it could possibly cause some latency issues when full histories are retrieved. Child workflows in that case can be used to partition the history.
Thanks @tihomir . I was going to implement the partitioning just like you mentioned, but I didn’t want to go to so much trouble just to call SignalExternalWorkflow Thanks for the details though.
Hi @maxim, I have a scenario where thousands of workflows get created during the day and all of them block on a signal name which is unique to those workflows. Then, at the end of the day, there is a trigger workflow that get’s kicked off which is supposed to unblock all of those earlier workflows by sending a signal which is unique to the workflow it is trying to unblock.
Sorry, this sounds like a solution, not a use case. What is the reason to use a signal which is unique for each workflow? Why, for example, not use the same signal name and different signal arguments?
What is the reason to use a signal which is unique for each workflow?
I guess I’m just working within some existing constraints. We also have a custom temporal UI which displays workflows blocked on a signal and provides the ability to send a signal to them via the custom temporal UI along with a fixed payload.
Why, for example, not use the same signal name and different signal arguments?
I’m curious whether this would somehow reduce the number of SignalExternalWorkflow calls? Also, is there some sort of a broadcast mechanism for signals?
There is no broadcast mechanism for signals yet.
But as I don’t understand the use case I cannot give you options. For example, you could use an activity to signal many workflows without growing the history.
Thanks for the details though.