Hey Guys,
A question pop up today, if temporal server overloaded, what should we do to purge the task queue?
1 Like
What do you mean by purging the task queue?
Do you mean you want to lose all open workflows? Then just wipe out and recreate the database.
AFAIK the majority of the use cases do not allow data loss and deal with overloads by slowing down the starts of new workflows and scaling up persistence and the cluster.
Ok only one queue i want to close all open workflows, is there command for that? Or we have to terminate all the workflows by api?
You can batch terminate workflows based on their properties. But it only works when Elastic Search integration is enabled.
Speak of the devil, for ES, would you guys start a doc section on how to hook it up and etc?
Similar requirement. I would like to purge a lot of workflows that are in extremely bad state and are waiting for an imaginary parent to open up again.
How do I just purge a task queue?
You batch terminate these workflows.
I understand the ideal situation is we would never want to terminate something, but reality is just different.
I just don’t want to do a reset on a lot of fast moving workflows especially given we don’t have bulk check pointed reset feature e.g.
Once we have sophisticated tools to diagnose and correct workflows at scale, this would be a reality.
But right now we need some sort of cyanide for the workflows. If possible, we should get such tools built into the workflow UI.
And try to get it done without ES also if possible, at least simple searches like name starts with, belonging to task queue etc.
Batch Terminate is what you want. But unfortunately, it only works with enhanced visibility. The good news is that very soon the enhanced visibility will be available in all SQL databases without ES.