I am trying to rate limit a certain activity (for example only one execution per second). For testing, all I do is print the current timestamp in that activity. I call the activity from a workflow, that does nothing else but call this activity. I made a seperate worker for that activity and the activity has it’s own task queue, which is connected to that worker. Now I set the parameter TaskQueueActivitiesPerSecond to 1, so only one activity can be executed per second. For testing I then started 20 workflows, which each just call this activity (there is a separate worker and task queue to handle the workflows).
Now I observe the following behavior. Instead of observing one execution per second, like the parameter would suggest, there are 2-10 concurrent executions, followed by a break longer than 1 second. How can I force the worker, to actually listen to the parameter and only poll one activity from the task queue per second and not exceed the set limit?
Thank you in advance for any help.
I’m having this issue too – is this by design, or is this a misconfiguration on my part?
Here’s a quick example app with the issue as well GitHub - vigneshv59/temporal-rate-limit-test. The output looks something like this:
Test activity 2023-12-07 22:42:20.938586 -0800 PST m=+13.477650834
Test activity 2023-12-07 22:42:20.953206 -0800 PST m=+13.492271209
Test activity 2023-12-07 22:42:20.965526 -0800 PST m=+13.504591042
Test activity 2023-12-07 22:42:20.994732 -0800 PST m=+13.533796667
Test activity 2023-12-07 22:42:24.946264 -0800 PST m=+17.485310709
Test activity 2023-12-07 22:42:24.963022 -0800 PST m=+17.502068626
Test activity 2023-12-07 22:42:24.995761 -0800 PST m=+17.534807417
Test activity 2023-12-07 22:42:25.021277 -0800 PST m=+17.560323834
...
Okay, seems like I figured out what was going on, the rate limit is set to be the rateLimit/nPartitions per task queue, so since the default is 4 partitions, it ends up sending 4 requests every 4 seconds instead of 1 request every second. Not sure if this is intended behavior or not, but setting the number of partitions to 1 fixes the issue (and I guess that’s sufficient for a low RPS anyways).
That indeed fixed it for me too, thanks a lot! For my cases the rates this can achieve is more than enough.