# Activity poller becomes inactive, activities stuck in PENDING\_ACTIVITY\_STATE\_SCHEDULED state

**URL:** https://community.temporal.io/t/activity-poller-becomes-inactive-activities-stuck-in-pending-activity-state-scheduled-state/16974
**Category:** Community Support
**Tags:** typescript-sdk
**Created:** [March 30, 2025, 5:15pm UTC](https://community.temporal.io/t/activity-poller-becomes-inactive-activities-stuck-in-pending-activity-state-scheduled-state/16974 "2025-03-30T17:15:18Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![mpontus](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/mpontus/32/6930_2.png) [@mpontus](https://community.temporal.io/u/mpontus)
#### Post date: [March 30, 2025, 5:15pm UTC](https://community.temporal.io/t/activity-poller-becomes-inactive-activities-stuck-in-pending-activity-state-scheduled-state/16974/1 "2025-03-30T17:15:18Z")

</div>

Hi everyone,

We have a workflow that periodically fetches a list of ids using an activity and starts a child workflow for each of them. The child workflows run a long-polling activity sending periodic heartbeats. Child workflows also respond to updates, making a request using activity and returning the response to the client.

Everything is working fine initially after deployment (using ECS with a single temporal server and one worker instance). However, after a while the main workflow stops executing activities, with the last activity stuck in `PENDING_ACTIVITY_STATE_SCHEDULED` state. The long polling activities continue to run and send periodic heartbeats, which are visible in the UI. If a child workflow receives an update, however, it will try to start an activity that will also get stuck in a pending state.

I can see in the Temporal UI the worker active as Workflow Task Handler but inactive as Activity Handler. Using the CLI, if I run `temporal task-queue describe` immediately after deployment, I can see active pollers for both workflow and activity task queues. After a while, the backlog begins to grow in the activity task queue and then the activity poller disappears.

The issue seems to resolve itself if the worker task is restarted. Then it picks up all pending activities, and everything returns to normal for a while. I could not find any log messages in the worker or temporal server logs that would indicate some error during the time of the incident.

Has anyone experienced a similar problem, or have any ideas on what may be causing an activity poller to disappear? Would appreciate any advice on how to diagnose this further.

---

<div class="post-metadata">

### Author: ![maxim](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/maxim/32/8_2.png) [@maxim](https://community.temporal.io/u/maxim)
#### Post date: [March 30, 2025, 6:55pm UTC](https://community.temporal.io/t/activity-poller-becomes-inactive-activities-stuck-in-pending-activity-state-scheduled-state/16974/2 "2025-03-30T18:55:34Z")

</div>

This is a common problem. Temporal SDK only polls when there are free slots to execute activities. If all slots are taken, polling stops. Make sure that your activity implementations release slots after completing. It can be exacerbated by activities running longer than the StartToClose timeout. The service sees the timeout and retries an activity according to the retry options even if the activity is still running on a worker.

---

<div class="post-metadata">

### Author: ![mpontus](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/mpontus/32/6930_2.png) [@mpontus](https://community.temporal.io/u/mpontus)
#### Post date: [March 31, 2025, 5:43pm UTC](https://community.temporal.io/t/activity-poller-becomes-inactive-activities-stuck-in-pending-activity-state-scheduled-state/16974/3 "2025-03-31T17:43:16Z")

</div>

Hi Maxim, thank you for your response.

I’m not sure if this description is applicable to our case. The issue can be reproduced with only a single child workflow running. In this setup, there will be at most three activities running at the same time.

Screenshots below demonstrate how the problem typically manifests:

- `fetchAccounts` activity is running in the main workflow at regular intervals until it eventually hangs
- `statPolling` The long polling activity in child workflows continues to run and send periodic heartbetas
- `getRawInventory` is short-lived activity started in response to updates sent to a child workflow

 ![image](https://us1.discourse-cdn.com/flex016/uploads/temporal/original/2X/7/741483c8b5d1fdf819fe0bef53831b3e01e1bd96.png)  
 ![image](https://us1.discourse-cdn.com/flex016/uploads/temporal/original/2X/9/9bd391105a753fe9dd2fcb19e686a1aca36227fe.png)

The long-polling activity is designed to run forever, sending periodic heartbeats and properly handling cancellation. The other activities have `startToCloseTimeout` set to a minute, but they usually finish in a couple of seconds and are not likely to run in parallel.

Given this context, could there be other factors at play here?

---

<div class="post-metadata">

### Author: ![maxim](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/maxim/32/8_2.png) [@maxim](https://community.temporal.io/u/maxim)
#### Post date: [March 31, 2025, 6:06pm UTC](https://community.temporal.io/t/activity-poller-becomes-inactive-activities-stuck-in-pending-activity-state-scheduled-state/16974/4 "2025-03-31T18:06:49Z")

</div>

Have you checked [worker\_task\_slots\_available](https://docs.temporal.io/references/sdk-metrics#worker_task_slots_available) metric?

---

<div class="post-metadata">

### Author: ![mpontus](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/mpontus/32/6930_2.png) [@mpontus](https://community.temporal.io/u/mpontus)
#### Post date: [April 4, 2025, 1:05am UTC](https://community.temporal.io/t/activity-poller-becomes-inactive-activities-stuck-in-pending-activity-state-scheduled-state/16974/5 "2025-04-04T01:05:03Z")

</div>

Thanks for the pointer, Maxim! Enabling the SDK metrics was invaluable help for diagnosing the issue. As you suggested, I was able to observe the number of used slots constantly growing before the activities froze.

 ![Screenshot From 2025-04-03 00-59-30](https://us1.discourse-cdn.com/flex016/uploads/temporal/original/2X/4/4484f6281567d5b881022c840fb583016ab1754b.png)

I narrowed down the problem to an activity from the main workflow that was meant to observe child workflow’s completion after the main workflow was restarted as new.

The change that was needed was to handle activity cancellation as follows:

```auto
import { cancelled } from '@temporalio/activity';

await Promise.race([
  workflowClient.getHandle(workflowId).result(),
  cancelled(),
]);

```

I now have an on how activity slots remain occupied when activity is retried. If activity returns a promise that never resolves, its slot will never be freed.

I think this idea is wrong because if the above code could be wrapped around every activity, that would be the default behavior. So there’s probably more going on under the hood that I’m missing.

Looking forward to finding more information on activity slots in the future. Thanks for all the pointers, Maxim.

---

<div class="post-metadata">

### Author: ![maxim](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/maxim/32/8_2.png) [@maxim](https://community.temporal.io/u/maxim)
#### Post date: [April 4, 2025, 6:39pm UTC](https://community.temporal.io/t/activity-poller-becomes-inactive-activities-stuck-in-pending-activity-state-scheduled-state/16974/6 "2025-04-04T18:39:38Z")

</div>

Most activities are not long-running and don’t handle cancellation. And cancellation requires heartbeating for delivery. So, wrapping all activities in this code is not going to work.

In the future, we are planning to add cancellation that doesn’t rely on heartbeating. Such a change might be considered.
