I think this problem can be generalised into the pub/sub pattern: on one end, I want to keep sending signals, on the other end, I consume the signal and process it, then return continue_as_new
to process another.
I don’t think the workflow.Signal is well suited for this problem. As @maxim explained in the other Q&A : Temporal is a control plane, not a data plane.
To workaround this issue, we used an external queue and then created a spacial “consumeMessage” activity that can wait indefinitely, and only consume one message and return.
IMHO, this is not a bad pattern, we fully utilised temporal’s capability to control the execution and we also have the data flowing through the activities without the restriction of history size.
One problem is if this continue_as_new
workflow repeats too many times, we need to garbage collect
them often.