Hi, where can I read more about the options for rate limiting?
I am interested in:
My understanding is that both are possible, but I must be missing something obvious.
Thank you,
Jacques
Just noticed that there is a property with a different name across Java and Go SDKs:
Use WorkerOptions properties to configure.
Use WorkerOptions properties to configure
This is exactly what I was looking for. Thank you Max!
Is the following configuration possible?
I have a worker that will run on special hardware (FPGA enabled). That worker can either do jobs of type A or B; however, it can only do one A job at a time, but it can do 10 B jobs at a time. The worker should only process jobs of one type at a time (i.e. no concurrent jobs of A and B on the same node).
How would this be configured?
With such requirements wouldn’t it lead to full starvation of job A?
A given workflow will have several tasks to accomplish, with a blend of types A and B. There will be dedicated activity queues for a single workflow, but only one worker node. I’m want to get a feel for how to run B jobs in parallel while ensuring A jobs run serially.
So there is a finite number of jobs that will ever go into the tasks queues.
If you have only one workflow execution (instance) using that worker then I would implement all the logic in the workflow itself. It has full control over which activities are scheduled and at which point.
- taskQueueActivitiesPerSecond - global task queue limit across all the workers
It’s probably the most practical to set task queue configuration on the worker level, since they are most closely coupled, but it seems odd for a single worker to be able to dictate the queue’s behavior.
What happens if 2 workers polling from the same queue set different values for taskQueueActivitiesPerSecond?
It’s probably the most practical to set task queue configuration on the worker level, since they are most closely coupled, but it seems odd for a single worker to be able to dictate the queue’s behavior.
We realize this and might add the ability to configure task queues through some admin APIs.
What happens if 2 workers polling from the same queue set different values for
taskQueueActivitiesPerSecond?
I believe the lowest value will be used. It is done to support updates to the value during worker deployments.
Facing higher activity per sec load, will the behavior throttling those activities or save them for later process using the max rate?
They will be saved in the task queue for later processing.
Do I need to delete the task queue for these changes to take affect? I made changes but I am not seeing the rate limiting take affect
No, task rate limiting is dynamic and can be updated any time by the worker.
Hi All,
In my case have multiple worker reading from task queue concurrently can i still use taskQueueActivitiesPerSecond for setting up global task queue limit for workflows .?
taskQueueActivitiesPerSecond is global across all workers polling on the task queue. Note that it applies to activities only, not workflows.
Do we have anything we can similarly apply to workflows .?
What are you trying to achieve? What is the use case for limiting the workflow execution rate?
So our workflows making calls to others services and we need to make call the these services within service limit. And i would like to handle this configuration at workflow level. In future planning to have multiple workflows which will handle different job
So
we need to make call the these services within service limit
You would need to make these 3rd party service calls from within activity code, think in this case rate limiting activities (and setting activity timeout options per your business needs) would still be what you need imo.