# History Service OOM exception

**URL:** <https://community.temporal.io/t/history-service-oom-exception/8865>\
**Category:** Community Support\
**Tags:** history, server\
**Created:** [July 17, 2023, 12:24pm UTC](https://community.temporal.io/t/history-service-oom-exception/8865 "2023-07-17T12:24:37Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dhanraj](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/dhanraj/32/1790_2.png) [@Dhanraj](https://community.temporal.io/u/Dhanraj)\
**Post date:** [July 17, 2023, 12:24pm UTC](https://community.temporal.io/t/history-service-oom-exception/8865/1 "2023-07-17T12:24:37Z")

</div>

Hi,

We are seeing lot of **OOM exceptions** in History service pods. We are currently running **3 pods** with each pod having **8gb** of memory.

We continue to see **pods getting restarted** when upper limit of memory is reached. Attached grafana dashboard for memory. **We are running temporal server v1.20.2.**

We continue to see below entries into history logs:

{“level”:“warn”,“ts”:“2023-07-17T01:50:07.050Z”,“msg”:“ **Critical attempts processing workflow task** ”,“service”:“history”,“component”:“shard-controller”,“address”:“10.1.1.207:7234”,“shard-id”:7998,“address”:“10.1.1.207:7234”,“wf-namespace”:“kairos-pp-main”,“wf-id”:“273230334”,“wf-run-id”:“61d50edd-a306-460d-a83c-e73eea7f3d34”," **attempt":407,** “logging-call-at”:“workflow\_task\_state\_machine.go:887”}

Looking at logs, it looks like some stale workflows are attempting retry and never gets executed and are retained in memory and getting piled up. Can this be the reason for memory leak?

Also, we have noticed that only one pod is doing heavy lifting and other pods remain stable. When the pod in question is killed, other pod does heavy lifting and cycle continues. Is this expected behaviour? How can we achieve even memory distribution?

 ![Screenshot 2023-07-17 at 12.28.46 PM](https://us1.discourse-cdn.com/flex016/uploads/temporal/original/2X/d/d6998641362773b4df49e93ce61a0c93b7a0073f.png)

Could you please help us troubleshoot this issue?

@Andrey_Dubnik@Kishore_Gunda @peeyushchawla @sidhu.sb - fyi

Thanks,  
Dhanraj