We are doing some experiments around the Temporal Java SDK client memory consumption. We also developed a small tool to keep track of the application process’s memory consumption and plot it
These experiments are performed in such a way: at the beginning of measurement period, we trigger the workflow which will run exactly 1 hour. After workflow completes, we won’t trigger anything else and just keep the measurement ongoing to see the performance of temporal client in the idle state
To set up a baseline, the application that we embed our client takes around 710 MiB of memory. Its memory consumption is very stable and won’t vary too much over the time.
In several experiments we observe one common patterns
- There is a significant increase of memory consumption happens around 1 hour after the completion of workflow. This can be indicated by the sudden steep increase of memory in the plot
For example,
and many more
In the last example, the workflow completes at 2024-02-16 10:22:04.77
Looking at the memory consumption, we can see a sudden increase occurs after 1 hour at 2024-02-16 11:23:12
from 756.83MiB to 767.37MiB.
We are looking for some potential explanation to understand why is always a sudden increase of memory consumption on the SDK client side one hour after last workflow completion? Understanding this behavior will help us better tailor the Temporal client based on our requirements. Thanks