Rate Limiting Based On Metadata

Community Support
1

Is there a recommended way to rate limit based on metadata like Account or User ID?

For example, given 10 customer accounts where 9 have 100 users each and 1 has 1000 users, what would be the best way to ensure each account is guaranteed at least 10% of worker capacity and no single user could monopolize all of an account’s reserved a worker pool’s capacity?

(Edited to hopefully improve clarity.)

1 Like
3

When you say “account” do you mean Temporal namespace? Or you mean that each account creates certain workflow types and you want to provide rate limiting based on a workflow attribute (which is account id in this case).

I don’t understand the use case to give concrete recommendations.

4

When I say Account I mean an organization, company, or department. In other words, conceivably a workflow attribute.

Imagine, for example, a parent workflow that starts an expensive child workflow for each user in the account. If multiple accounts start the parent workflow near the same time, an account with a much larger number of users could monopolize all available workers by simply being first to queue up a large number of activities associated with each user. When there’s no contention, it’s desirable for an account to be able to use 100% of available workers but as soon as there’s contention between accounts it’s desirable to attempt fair scheduling in order to keep latency proportional to account or workflow size.

Obviously it would be possible to partition capacity by creating separate queues for each account, but the result is either potentially significant idle capacity or latency waiting for workers to scale out.

5

I see. Unfortunately we don’t support anything like this at this point without using some external task storage.

Here are some strawman feature ideas to add to the Temporal:

  • priority task queues. In this case the activity and workflow tasks from each department can have different priorities. The problem is that it doesn’t help from high priority tasks starving the lower priority ones.
  • Priority task queues with probabilistic priorities. This would solve your problem out of the box.
  • Semaphores. We could add a shared semaphore abstraction. It would allow to gate the number of activity invocations based on whatever criteria application needs.

@ryland I think all of the above deserves to be added to feature backlog.

6

Thanks!

If nothing else it’s nice to have confirmation I’m not missing something or being insufficiently creative. :wink:

7

Hello @maxim, did this feature been added to your backlog? I’m not asking for your roadmap but just wanted to know if it had been noted somewhere on your side.
We are really interesting by this feature.

Thanks!

1 Like
8

I found this discussion after searching for ways to solve prioritization with task queues. I’m interested too to know if there is any way that a feature for “Priority task queues with probabilistic priorities” may be added to the backlog. If it’s out of scope for Temporal that would be nice to know as well.

9

Please file an issue against the Temporal repo to get this request tracked. We think it is in scope, but I doubt we will have cycles to work on this in the near future.