I am trying to figure out if I can implement some type of flexible rate limiting in Temporal IO. For that I would like to find out, what workflows of a certain type including a certain parameter value have executed successfully during the last x minutes. Is there a possibility to do this? I would like to do that out of an activity. I am using the Go SDK
Thanks in advance!
find out, what workflows of a certain type including a certain parameter value have executed successfully during the last x minutes
You could use visibility to query workflow executions, something like
ExecutionTime BETWEEN "2023-10-26T00:00:00+01:00" AND "2023-10-26T00:00:00+01:00" AND WorkflowType="YourType" AND ExecutionStatus="Completed"
a certain parameter value
and create a custom search attribute to add to the query
Note that visibility store is eventually consistent
some type of flexible rate limiting
can you provide more info about the use-case?
Temporal allows you to rate-limit the number of activities started per second, per taskqueue, along with other configuration
TaskQueueActivitiesPerSecond internal package - go.temporal.io/sdk/internal - Go Packages
flexible rate limiting, I mean that the time frame and the amount can be given flexibly. So I am not bound to “x per second” but can say “x per timeframe y”, like “4000 per 6 hours”. By querying the amount of executions in the last timeframe y I think I could implement it, if it is performant enough.
Thanks for the answer, I’ll check out the visibility feature!
I have a question to the visibility. If I have two namespaces, will I only receive results of executions in that namespace?
I am also facing another difficulty. When starting 20 Executions back to back I also want to find out how many executions are running already. Therefore I changed
(ExecutionStatus='Completed' OR ExecutionStatus='Running'). But for example for the first 15 executions it gives me 0. Is there a way to fix this, that actually the first one shows zero, the second one 1, etc? The executions are not started simultaneously but just right after each other in a for loop, but I can still observe this behaviour.
With kind regards