# Timeouts on visibility-tasks but workflows still completing

**URL:** <https://community.temporal.io/t/timeouts-on-visibility-tasks-but-workflows-still-completing/6879>\
**Category:** Community Support\
**Created:** [January 9, 2023, 9:41am UTC](https://community.temporal.io/t/timeouts-on-visibility-tasks-but-workflows-still-completing/6879 "2023-01-09T09:41:09Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Francis\_Rooney](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/francis_rooney/32/3051_2.png) [@Francis\_Rooney](https://community.temporal.io/u/Francis_Rooney)\
**Post date:** [January 9, 2023, 9:41am UTC](https://community.temporal.io/t/timeouts-on-visibility-tasks-but-workflows-still-completing/6879/1 "2023-01-09T09:41:09Z")

</div>

Seeing a lot of errors when running around 10k workflows all in short succession. DB load doesn’t go above 60% so don’t believe its a performance issue there at least. The errors in the logs of the history service look like.

`visibility task 21~47186285 timedout waiting for ACK after 1m0s`

Unsure if its related - but the matching service is also throwing a lot of errors that look like the following.

`"error":"Workflow task not found."`

Again - everything eventually completes fine, but it is taking longer than I’d expect. Wonder if anyone has any suggestions on tracking down the root cause.

---

<div class="post-metadata">

**Author:** ![maxim](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/maxim/32/8_2.png) [@maxim](https://community.temporal.io/u/maxim)\
**Post date:** [January 9, 2023, 6:26pm UTC](https://community.temporal.io/t/timeouts-on-visibility-tasks-but-workflows-still-completing/6879/2 "2023-01-09T18:26:38Z")

</div>

The visibility task pushes data to Elastic Search. It looks like your ES is underpowered.

---

<div class="post-metadata">

**Author:** ![tihomir](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/tihomir/32/6580_2.png) [@tihomir](https://community.temporal.io/u/tihomir)\
**Post date:** [January 9, 2023, 7:08pm UTC](https://community.temporal.io/t/timeouts-on-visibility-tasks-but-workflows-still-completing/6879/3 "2023-01-09T19:08:32Z")

</div>

Just to add, for visibility you can measure visibility latencies via server metrics that could be good indication on possible issues:

`histogram_quantile(0.95, sum(rate(task_latency_bucket{operation=~"VisibilityTask.*", service_name="history"}[1m])) by (operation, le))`

as well as similar query for the `task_latency_queue_bucket` metric with which you can see how long visibility tasks are sitting in queue waiting to be processed.

Regarding `"error":"Workflow task not found."` this can be a transient error (so not affect your workflow executions) but would check your matching service cpu utilization as well.

---

<div class="post-metadata">

**Author:** ![Francis\_Rooney](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/francis_rooney/32/3051_2.png) [@Francis\_Rooney](https://community.temporal.io/u/Francis_Rooney)\
**Post date:** [January 11, 2023, 10:35am UTC](https://community.temporal.io/t/timeouts-on-visibility-tasks-but-workflows-still-completing/6879/4 "2023-01-11T10:35:16Z")

</div>

Thanks - seemed to be this! Was using the count-api to see my workflows and that was going very slowly - didn’t register that was powered by elasticsearch so temporal itself was working fine!