I’d like to monitor Temporal metrics and automatically scale my Temporal workers based on the number of workflows in the task queue. My goal is to increase the number of workers if the queue has a high number of workflows waiting to be processed and to scale down if there are fewer workflows in the queue. However, I haven’t found a direct metric that specifically tracks the number of pending tasks in the queue for workers to pick up. Given this, what alternative methods are suggested to scale workers up and down based on task queue load?
Take a look at task queue backlog information in docs here. My understanding is that there are
plans to expose this as a worker metric at some point, but currently yeah you’d need to poll
to get backlog info.