I have services that use Python with Temporal, and in all of them RAM memory keeps accumulating. It keeps growing until it exhausts the pod or Docker memory. We updated Python and the libraries, used the garbage collector, and nothing solved it. We even commented out all the code and left only one variable receiving a random value to see whether it could be something in the code, but even then the memory kept increasing. What could this be? Is there any Temporal or Python setting that could be causing this?
Below is an image of a Docker container that has not received anything to process for hours, but the memory is stuck at 732 MB and does not go down even with nothing to process. If I create new workflows, it only keeps growing more and more until the container runs out of memory. These tests were done in both Docker and Kubernetes, and the result was the same.
At the beginning the Python version was 3.10 now we are using 3.12
Temporal version: 1.23.0