# Concurrency limit for activities which call async APIs **URL:** \ **Category:** Community Support\ **Tags:** python-sdk\ **Created:** [August 14, 2026, 10:15pm UTC](https://community.temporal.io/t/concurrency-limit-for-activities-which-call-async-apis/19775 "2026-08-14T22:15:06Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![tempora](https://avatars.discourse-cdn.com/v4/letter/t/a587f6/32.png) [@tempora](https://community.temporal.io/u/tempora)\ **Post date:** [August 14, 2026, 10:15pm UTC](https://community.temporal.io/t/concurrency-limit-for-activities-which-call-async-apis/19775/1 "2026-08-14T22:15:06Z")
Hi there, I’m new to Temporal so please bear with me. I’ve got a workflow which fans out to many activities, all of which call a 3rd party API. The API asynchronously starts a job and returns. The activity currently returns a result and then the workflow is signalled when the job has completed via SQS. The issue is that the API has restrictions on 1) RPM and 2) in flight jobs. I’ve created a task queue, which my workflow pushes to and then workers pull from to execute the activity, this seems to solve 1) but I’ve still got the issue of in flight jobs. I’m using a global count of inflight jobs which gets incremented when the API is called and decremented when a job completes. I use this to limit when I push to the queue but it’s not ideal and it feels like there should be a more “temporal native” way of doing this. If nothing else I believe I can refactor to async activities which will allow the activities to be signaled on completion directly(?) but I don’t believe there’s a way to limit the number of in progress async activities. I know that `max_concurrent_activities` exists but I believe this is purely the number of concurrent activities executing, not including async activities that are idle. So my question is, is there anything I can do, using native Temporal constructs, that would allow me to limit the number of active activities that are pulling from the queue? E.g. the queue has 10,000 tasks in, workers drain the queue and wait for async activities to complete, but the number of workers executing+waiting doesn’t exceed N. Failing that, is there a better pattern I could be using? Thanks! ---
**Author:** ![cory.finger](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/cory.finger/32/8442_2.png) [@cory.finger](https://community.temporal.io/u/cory.finger)\ **Post date:** [August 17, 2026, 11:51pm UTC](https://community.temporal.io/t/concurrency-limit-for-activities-which-call-async-apis/19775/2 "2026-08-17T23:51:59Z")
Hey, Tempora! Welcome to the community! Rate limiting downstream services is a great question, and your situation throws the extra curve-ball of limiting the concurrency of asynchronous downstream APIs! Temporal can definitely help here and we provide several design patterns based on exactly which kind of concurrency limits fit your use-case best. I’d like to ask a few clarifying questions to ensure I can point you down the right path for your situation. - What are the exact rate & concurrency limits of the downstream API, if you wouldn’t mind sharing? - What is the expected load of your fanout process and is it known how many of these Workflows might be executing at the same time? How many activities might one Workflow execution start? How long does it usually take one of these activities to complete and the workflow to receive the resulting signal? - What should happen in the case of one of these response signals never firing or an activity failing? Should the workflow wait for the activity to complete before it exits? Should something happens in the case of the downstream API failing to return a response in a certain amount of time? - Are you trying to limit these things (rate limiting, concurrency) at a global level, per-worker level, or would workflow-level concurrency limits like it sounds like you use now, suffice? We can provide solutions for each of these types of limits, but they each have their own tradeoffs. - In the case of these rate limits do get hit, say from an external factor like a second process hitting this API, what should the worker/activity do in response to the (429 status?) they receive? - It sounds like you currently start an activity and then wait for a resulting Signal - is this purely for the purpose of rate limiting or does the Workflow utilize the response or knowledge the activity completed, in some way? How does the resulting service know which workflow to send the signal to? Let me know if any of these questions don’t make sense for your use-case! I’d also be incredibly curious to know if you’ve yet tried our [Async Activity Completion](https://docs.temporal.io/design-patterns/delayed-callback#pattern-3--async-activity-completion) feature as an alternative to the Activity → Signal approach, as that seems like it’d result in easier-to-read Event Histories while you’re trying different concurrency architectures out! ---
**Author:** ![tempora](https://avatars.discourse-cdn.com/v4/letter/t/a587f6/32.png) [@tempora](https://community.temporal.io/u/tempora)\ **Post date:** [August 18, 2026, 10:08am UTC](https://community.temporal.io/t/concurrency-limit-for-activities-which-call-async-apis/19775/3 "2026-08-18T10:08:39Z")
Hey Cory, thanks for your response. - So the RPM for the service is 25,000 (which we don’t intend to saturate) and the max concurrent jobs is 20,000 - The expected load is that we would have around 5 workflows per minute, fanning out to 2500-5000~ activities per minute. The initial call before `raise_complete_async` takes milliseconds, the wait for the external job to complete is around 30 seconds (though not guaranteed and we’re yet to test this API at real scale) - Before the call we write to a persistent store to record that we’re waiting for a result, we then have a separate workflow which checks for old records and cancels that activity if they haven’t been completed in a certain period - Workflow-level limits is fine. I’d be interested to hear the solutions for this, presumably it’s easier to implement this with a single dispatcher rather than several activity workers? - Once a task is pulled from the task queue and an activity is started, it attempts to call the API with its own exponential backoff policy. If this continues to fail then we mark the activity as failed and let the rest of the workflow continue. - The purpose is for rate limiting and to avoid polling the API directly. The API call sends a message to SQS on completion, a process then polls SQS for messages and from that message it builds a deterministic activity ID. Using that ID it then completes the activity. Thanks for the link, would you be able to clarify the exact difference between that and using `raise_complete_async` combined with `handle.complete()` ? Thanks again for the help ---
**Author:** ![cory.finger](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/cory.finger/32/8442_2.png) [@cory.finger](https://community.temporal.io/u/cory.finger)\ **Post date:** [August 18, 2026, 7:26pm UTC](https://community.temporal.io/t/concurrency-limit-for-activities-which-call-async-apis/19775/4 "2026-08-18T19:26:38Z")
Hey Tempora, thanks for those details! From hearing your approach and those details, I think you’re essentially doing the workflow-level rate-limiting in the suggested way! It also sounds like you’re using Async Activity Completions already (from your comments on `raise_complete_async`) and thus should have the best representation of the activities and their execution times in the Event History. One thing I haven’t heard you mention is the `max_task_queue_activities_per_second` configuration for rate limiting, which you might be interested in ([documentation link](https://docs.temporal.io/design-patterns/downstream-rate-limiting#comparison-with-alternatives)). Unfortunately you’re correct in thinking that the asynchronous nature of the activities, prevents them from being tied to `max_concurrent_activities` which is more aligned around preventing the worker from getting overloaded with concurrent work than it is around concurrency limiting that executes in an external system. For all these suggestions I’ve given an example of in an attached Python file so you can have a concrete script to look through! As it seems like you have the suggested setup for what you’re after, I’ll provide some notes that might help you avoid common pitfalls: - If you’re only using them for failure recovery, you **might** be able to avoid having a secondary workflow & persistence store to detect failures, by utilizing `start_to_close_timeout` in the ActivityOptions for the activity, paired with a `retry_policy` option that will externalize the retry logic from within the Activity. In doing so, it will allow you to set the `start_to_close_timeout` to **30 seconds** (or whatever you’d prefer). Which will naturally, durably fail the activity if no matching completion is received within 30 seconds without the reliance on a secondary system. In this case, Temporal Server becomes the secondary system that ensures this behavior! - Even without this change, you’ll want to make sure that **your Activity is** **idempotent** , and is resilient to being rerun. It is possible that this requirement will prevent you from getting rid of the secondary persistent storage if the downstream API does not offer idempotent requests. Overall this idempotency is necessary to both prevent the activity retries from causing unexpected behaviors and also ensure that the workflow remain resilient to replays in the case of a crash or other network interruption. - It’s useful to be aware of our [Per-Workflow Concurrency Limits](https://docs.temporal.io/cloud/limits#per-workflow-execution-concurrency-limits) around incomplete Activities, Signals and similar to avoid a common pitfall these high-concurrency cases can run up against when placed in production. I imagine with those rates you discussed, you’re already aware of them - but wanted to link them to be sure! - I’ll link you to our [Design Patterns](https://docs.temporal.io/design-patterns) guide and specifically our [Batch Processing Patterns](https://docs.temporal.io/design-patterns#batch-processing-patterns) - what you’ve implemented here I believe is very close to or otherwise the same as our “Sliding Window Pattern”! The benefits of having the **timeout and retry policy** be external to the activity itself and configured through Temporal, would include: - An accurate Event History that represents completions, failures and retries - A durable guarantee of the timeout logic, even in the case of a full worker crash or other system failure - An accurate view of execution times, that can then be monitored and alerted on via our [SDK Metrics](https://docs.temporal.io/references/sdk-metrics) I have linked both the script with simulated failures that implements everything mentioned above and a screenshot of what the resulting workflow timeline looks like with the retries externalized into the ActivityOptions! **Script Example:** [async\_api\_concurrency.py](https://gist.github.com/polymorfiq/423ad1e170f19699d342fc3fd8a0b0c0) ![Screenshot 2026-08-18 at 11.21.27 AM](https://us1.discourse-cdn.com/flex016/uploads/temporal/original/2X/7/75445cd89b18550570f273c7966e9a0bee987110.png) Let me know if you have any more questions, and best of luck with your use-case! ---
**Author:** ![tempora](https://avatars.discourse-cdn.com/v4/letter/t/a587f6/32.png) [@tempora](https://community.temporal.io/u/tempora)\ **Post date:** [August 18, 2026, 8:11pm UTC](https://community.temporal.io/t/concurrency-limit-for-activities-which-call-async-apis/19775/5 "2026-08-18T20:11:42Z")
Thanks for the super detailed answer! I am indeed using `max_task_queue_activities_per_second` but thanks for the heads up. - Good point about `start_to_close_timeout`, I’ll look into that - I hadn’t spotted the ‘batch processing patterns’ docs actually so that’s very helpful Having glanced at the script and the design patterns page, I think the disconnect for me is around how you keep track of in-flight jobs. I’ve been assuming that I needed something more persistent, e.g. a counter in RDS/Redis which I increment when I pick a task from the queue and decrement when the API completes. But looking at your example, the count is just in-memory, is this sufficient? My assumption had been that this might not be resilient enough in the case of things like Temporal workers restarting etc, or is that not the case? I will take a more in-depth look at the script though and make sure I’m following it correctly. ---
**Author:** ![cory.finger](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/cory.finger/32/8442_2.png) [@cory.finger](https://community.temporal.io/u/cory.finger)\ **Post date:** [August 18, 2026, 10:28pm UTC](https://community.temporal.io/t/concurrency-limit-for-activities-which-call-async-apis/19775/6 "2026-08-18T22:28:31Z")
Great question - the in-memory counter is sufficient, specifically because of the durability guarantees that make Temporal special - [Durable Execution](https://temporal.io/blog/what-is-durable-execution)! You might find [this Python-oriented walkthrough](https://docs.temporal.io/encyclopedia/event-history/event-history-python) particularly helpful in learning about the event history that powers Temporal’s replay functionality. A very short version of this is that, if the worker crashes and restarts, it loads its inputs and previous event history (all the activities and steps that it has executed until that point). So long as you follow the Temporal axiom of making your Workflow deterministic based on its inputs, your `workflow.execute_activity` loop will execute in the exact same order as it did in the crashed run. But **this time** , rather than actually executing the already-completed runs, it will utilize their [Event History](https://docs.temporal.io/workflow-execution/event#activity-events) to “replay” their outputs, rather than executing the code within the activity’s function. It will similarly automatically fulfill the relevant Futures they produce (for things like `asyncio`). In this way, your in-memory counter will reach a valid & representative state, before it starts hitting activities that have not been completed and starts executing their code and hitting the downstream API. One thing you do want to ensure is determinism in the Workflow logic as well as [Activity Idempotency](https://temporal.io/blog/idempotency-and-durable-execution) to handle those cases in which the Activity might be executed multiple times (say, in the timing of failures, crashes, loss of network connection). Through ensuring those things, you can benefit from the durability guarantees above! Below, I have a screenshot of how the workflow example I posted above reacts if I place a worker crash in the middle of it! It picks up right where it left off! ![Screenshot 2026-08-18 at 3.14.57 PM](https://us1.discourse-cdn.com/flex016/uploads/temporal/original/2X/8/85c405a1dc32c9475e00f7a5481c51ff1504fb0a.png) ---
**Author:** ![tempora](https://avatars.discourse-cdn.com/v4/letter/t/a587f6/32.png) [@tempora](https://community.temporal.io/u/tempora)\ **Post date:** [August 19, 2026, 7:39pm UTC](https://community.temporal.io/t/concurrency-limit-for-activities-which-call-async-apis/19775/7 "2026-08-19T19:39:50Z")
Thanks so much for all of this, it’s been super helpful! Also, I’m very impressed with Temporal, it’s a great product you guys have built. ---
**Author:** ![tihomir](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/tihomir/32/6580_2.png) [@tihomir](https://community.temporal.io/u/tihomir)\ **Post date:** [August 19, 2026, 8:18pm UTC](https://community.temporal.io/t/concurrency-limit-for-activities-which-call-async-apis/19775/8 "2026-08-19T20:18:01Z")
use taskQueueActivitiesPerSecond on task queue for task queue dispatch limit (note also there is a burst factor per tq partition if rpm limit is strict) dont complete activities right away - make downstream call then in same activity poll sqs for completion, activity should heartbeat, set proper activity timeouts control total concurrent activites across activity workers on tq by num\_workers \* max\_concurrent\_activities ---
**Author:** ![tempora](https://avatars.discourse-cdn.com/v4/letter/t/a587f6/32.png) [@tempora](https://community.temporal.io/u/tempora)\ **Post date:** [August 20, 2026, 4:20pm UTC](https://community.temporal.io/t/concurrency-limit-for-activities-which-call-async-apis/19775/9 "2026-08-20T16:20:27Z")
Hey just to follow up on this, am I right in thinking `_counter` in your example is a per workflow counter right? So if I have 10 concurrent workflows running then `counter=API_CONCURRENCY_LIMIT/10` ? The problem with this is that workflows aren’t balanced, therefore one workflow might need to invoke the API 500 times whereas another might only need 50. Is there a way to have a durable, in-memory counter across workflows? Thanks! ---
**Author:** ![tihomir](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/tihomir/32/6580_2.png) [@tihomir](https://community.temporal.io/u/tihomir)\ **Post date:** [August 20, 2026, 4:59pm UTC](https://community.temporal.io/t/concurrency-limit-for-activities-which-call-async-apis/19775/10 "2026-08-20T16:59:40Z")
Correct, per-wf counter dont impose a global cap so for the “x-workflows counter” you then need to use custom db maybe or a single or small pool of long-running executions that control limiting and you funnel others through them, but it becomes increasingly difficult to impl requirements, deal with deployments, for this impl on large scale That one of main reasons why added that earlier response. That alternative approach i mention would imho allow use case to scale and move bottleneck to “worker\_count × max\_concurrent\_activities” which is more forgiving. It also by not having to rely on async completion, give ability to do things like use activity apis (pause, reset, …), deal with cancelation and heartbeating easier, avoid deduplication of sqs signal deliveries etc Another thing i wanted to add just for reference regarding the code example shown - activity timeout (managed by temporal server) do not stop the activity work (on your worker) to continue executing until it completes or fails (or heartbeats). So i think a timeout on one job, in that sample, can trigger next job, while previous is still running. This would also, just like the issue you found, the per-wf counter, not fit with your use case requirements. You’d have to in this case add activity heartbeats on top of the previously mentioned things and heartbeat from external completion client which makes things a bit more difficult to do reliably at scale