Latency Metric that excludes sleep or idle time cases

Hi,
We’re looking forward to measure latency in a manner that reflects only the area where a client sends a request and its processing is started or it is being put in task queue. I suppose, anything of sort where end to end latency would be calculated wouldn’t be useful for us since it would also reflect idle time in cases such as long running workflows etcetera.

We used temporal_workflow_task_schedule_to_start_latency_seconds_bucket and service_latency but soon realized that it would not work since there were instances where some requests would fall under infinity bucket in case of temporal_workflow_task_schedule_to_start_latency_seconds_bucket.

My next bet would be on activity_schedule_to_start_latency. Any thoughts or other suggestions ?

Thanks,
PC

We’re looking forward to measure latency in a manner that reflects only the area where a client sends a request and its processing is started or it is being put in task queue.

You could look at service latencies (server metrics):

histogram_quantile(0.95, sum(rate(service_latency_bucket{service_type="frontend"}[5m])) by (operation, le))

and can add additional filter for specific operations such as “StartWorkflowExecution” and worker poll specific “RespondActivityTaskCompleted”, “RespondWorkflowTaskCompleted”.

SDK schedule to start latencies measure the time from when a workflow/activity task is placed on your matching host task queue until one of your workers successfully polls it to process.

Is there any latency metric which would just get the time
From - client starts
and
To - its placed to task queue.

Thanks

On the server metrics side, service_latency_bucket latencies are measured from when the Temporal frontend service receives the client request. Latencies of client requests to when frontend service receives this are not included as Temporal server does not control that part.

For communication latencies between your SDK apis and service, SDK metrics expose:
temporal_request_failure
temporal_request_latency
temporal_long_request_failure
temporal_long_request_latency

if that helps.