Limit concurrent database queries across workflow using mutex?

Community Support
1

Hello,

We are looking to be able to limit the number of concurrent queries to a database across workflows. I think this would be similar to the mutex workflow example but instead of only one lock on the resource, I want to allow a configurable amount like 5 or 10 queries can run concurrently on the database from any workflow. Any suggestions on how to do that? Is mutex still the right approach? and if so, what would the mutex lock workflow look like? I can only guess that the mutex workflow would have an array of currently held locks but I’m not sure how the channel management code would look like (I’m also new to Go). Is it possible to only pull from a channel if the size of an array is less than a set amount?

Another caveat: there are actually a dynamic number of these databases and they will have different restrictions on how many queries can run concurrently. We also will be dynamically creating these databases. I’ve previously looked into using task queues per database as resource management, but the dynamic nature of the databases prevent this option from being used.

2

What is the maximum rate of requests per second per mutex instance you need?

3

I’m not sure of the max rate but these larger queries so they generally take a few seconds to run (up to a few minutes for larger datasets).

4

The duration of queries doesn’t matter. A single workflow has limited throughput. So if you use a mutex workflow pattern make sure that it doesn’t need to process more than a few signals per second.

Otherwise, you can rewrite the mutex to support the semaphore-like behavior.