Temporalite is similar to auto-setup as in it starts all service roles in the a single process. This makes it imo hard to find out which service would need to be looked at (especially without server metrics).
My best guess would be to try to look at dynamic config settings for history service cache, see more info here.