startToCloseTimeout not terminating execution of activity

Community Support
1

This is for the typescript SDK (if that matters).

I’ve got a workflow, call it importForCustomer which is implemented as one activity, and that activity typically runs for ~15m. The activity is using the following ActivityOptions:

{
  startToCloseTimeout: '30 minute',
  retry: {
    maximumAttempts: 1,
  },
}

I’ve also got a parent workflow, call it importForAllCustomers. This runs periodically via a temporal cron job, and it simply starts a child importForCustomer workflow for each customer:
await startChild(importForCustomer(customerId)).

The problem: one customer’s import always takes longer than 30m and temporal fails it with:

ActivityTaskTimedOut: activity StartToClose timeout
WorkflowExecutionFailed: Activity task timed out

However, my logging indicates that it continues to run on the worker until it completes. What’s worse, another instance of the workflow starts running at the same time when the next temporal cron job starts (since temporal thinks the workflow failed, but the worker seems to be letting it run to completion). It is worth calling out that the activity is not one long blocking operation, it executes many async functions within the activity, so it’s not that it wouldn’t be able to respond to a request to cancel if temporal is internally doing something like that.

Is this expected behavior? Should I be doing this differently?

2

Hi @Shawn_Jones

yes the activity will keep running unless you heartbeat from it, and subscribe to cancellation.

We have some documentation here Namespace: activity | Temporal TypeScript SDK API Reference

let me know if it helps,
Antonio

3

Thanks so much!

As an aside, it feels like this should be noted next to startToCloseTimeout docs :slight_smile: