HI
I am getting WorkflowTaskTimedOut, when running a large number of flows. Any inputs please!
attaching the screenshot
Can you provide more information?
One common case where WorkflowTaskTimedOut event is logged is when a worker restarts / is not available. Looking at the logs you might not have a worker running that has the child workflow registered.
There could be other possible issues, but hard to tell without more info.
Thanks @tihomir, will fetch more info. How ever when i run like 100 flows it does not happen
Hey @Ramprasad_Indarapu,
If workflow tasks are timing out it typically points to some scalability issue in your setup. In the example you posted above it seems to be ScheduleToStart timeout. This typically means the bottleneck could be on the workflow polling for workflow tasks. Can you try to run your scenario with multiple workers to see if that helps?
Here are some other posts which explains workflow tasks and how to interpret timeouts:
Hi @jackban
Can I ask why you need to set the WorkflowTaskTimedOut> 10 seconds?
WorkflowTask is meant to take less than 10 seconds, maximum value allowed by the server is 1 minute.
WorkflowTaskTimeOut type startToClose can happen if your worker crash during the workflow task execution, in this case, another worker can pick up the workflowTask and continue the workflow execution.
It can also happen if your workflow code executes a heavy operation that takes more than 10 seconds. Is this your case?
Hi @antonio.perez
We have this issue we have heavy operation (in th actvity not in the workerflow) which can takes more than 10 secods and its really CPU hunger actvity , and we are facing the WorkflowTaskTimedOut, is the same case ?
- i can also see the possible deadlock warning the worker pod logs.
- we dont have any limitation on the worker pod cpu usage but we starts with 0.1 (request cpu 100m) cpu power but it can use what ever it wants.