# WorkflowTaskTimedOut observed during performance testing

**URL:** <https://community.temporal.io/t/workflowtasktimedout-observed-during-performance-testing/10193>\
**Category:** Community Support\
**Tags:** python-sdk\
**Created:** [November 25, 2023, 10:51am UTC](https://community.temporal.io/t/workflowtasktimedout-observed-during-performance-testing/10193 "2023-11-25T10:51:13Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![gs\_981](https://avatars.discourse-cdn.com/v4/letter/g/9f8e36/32.png) [@gs\_981](https://community.temporal.io/u/gs_981)\
**Post date:** [November 25, 2023, 10:51am UTC](https://community.temporal.io/t/workflowtasktimedout-observed-during-performance-testing/10193/1 "2023-11-25T10:51:13Z")

</div>

We are observing WorkflowTaskTimedOut errors when multiple workflows are triggered at once.  
we are using a parent and child workflow and these timeout errors are being observed in both parent and child.  
workflow code is a simple code calling list of multiple activities.  
Only piece of executable work we are doing in parent workflow is to get the workflow started time using  
start\_date\_time = workflow.now().strftime(‘%Y-%m-%d %H:%M’)  
end\_date\_time = (datetime.strptime(start\_date\_time, ‘%Y-%m-%d %H:%M’) + timedelta(hours=2))   
.strftime(‘%Y-%m-%d %H:%M’)

we have not set any timeout and hence defaults are used.

Please help understand if we missing any set up.

---

<div class="post-metadata">

**Author:** ![tihomir](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/tihomir/32/6580_2.png) [@tihomir](https://community.temporal.io/u/tihomir)\
**Post date:** [November 27, 2023, 2:37am UTC](https://community.temporal.io/t/workflowtasktimedout-observed-during-performance-testing/10193/2 "2023-11-27T02:37:06Z")

</div>

Can you share full error(s) you are getting please?

Do you have sdk (worker) metrics configured? If not check samples like [https://github.com/temporalio/samples-python/tree/main/prometheus](https://github.com/temporalio/samples-python/tree/main/prometheus) for more info.

From sdk metrics look at  
workflow\_task\_execution\_latency  
workflow\_task\_replay\_latency  
request\_latency  
request\_failure

---

<div class="post-metadata">

**Author:** ![tihomir](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/tihomir/32/6580_2.png) [@tihomir](https://community.temporal.io/u/tihomir)\
**Post date:** [November 27, 2023, 1:27pm UTC](https://community.temporal.io/t/workflowtasktimedout-observed-during-performance-testing/10193/6 "2023-11-27T13:27:31Z")

</div>

> NotFound, message: “Workflow task not found.”

This is expected when the timed out workflow task response is sent by your worker.  
The WorkflowTaskTimedOut timeout type is StartToClose meaning your worker is not able to respond its completion within the default timeout of 10s.

Still think it’s important to check the earlier mentioned sdk (worker) metrics as well as check your worker pods/containers cpu and mem utilization during this time. This would allow you to pinpoint something rather than guessing.

---

<div class="post-metadata">

**Author:** ![gs\_981](https://avatars.discourse-cdn.com/v4/letter/g/9f8e36/32.png) [@gs\_981](https://community.temporal.io/u/gs_981)\
**Post date:** [November 28, 2023, 9:48am UTC](https://community.temporal.io/t/workflowtasktimedout-observed-during-performance-testing/10193/7 "2023-11-28T09:48:21Z")

</div>

When ran in debug mode saw this error in the logs just before the workflow tasksTimed out.

DEBUG:temporalio.worker.\_workflow:Evicting workflow with run ID e5af6845-296d-4b15-a936-dc5bdd2274e1, message: Error reporting WFT to server  
DEBUG:temporalio.worker.\_workflow:Evicting workflow with run ID cb857635-1cac-4562-ab1e-8ee1a85d7a9f, message: Error reporting WFT to server

any info on this error is appreciated.

---

<div class="post-metadata">

**Author:** ![tihomir](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/tihomir/32/6580_2.png) [@tihomir](https://community.temporal.io/u/tihomir)\
**Post date:** [December 1, 2023, 5:40pm UTC](https://community.temporal.io/t/workflowtasktimedout-observed-during-performance-testing/10193/9 "2023-12-01T17:40:02Z")

</div>

Could you look at your worker metric  
`request_failure`

you can filter it by operation and status\_code. Look if you see any errors reported for operation RespondWorkflowTaskCompleted and see the status code.