Hi,
We upgraded Temporal self hosted cluster to version v1.25.2 from v1.19.1. Previously, we used to see temporal server metric ‘task_schedule_to_start_latency’ for task_type ‘Workflow’ and ‘Activity’ both with version v1.19.1 but with version v1.25.2, only task_type ‘Workflow’ metric is seen and not ‘Activity’ one.
In 1.19.1 - For Activity tasks in /service/history/api/recordactivitytaskstarted/api.go
metrics.GetPerTaskQueueScope(
taggedMetrics,
namespaceName.String(),
taskQueueName,
enumspb.TASK_QUEUE_KIND_NORMAL,
).Timer(metrics.TaskScheduleToStartLatency.GetMetricName()).Record(
scheduleToStartLatency,
metrics.TaskQueueTypeTag(enumspb.TASK_QUEUE_TYPE_ACTIVITY),
)
While in 1.25.2 - For Activity Tasks (in recordactivitytaskstarted/api.go
-
metrics.TaskScheduleToStartLatency.With(
workflow.GetPerTaskQueueFamilyScope(
taggedMetrics,
namespaceName,
ai.GetTaskQueue(),
shard.GetConfig(),
),
).Record(scheduleToStartLatency)
Looking at the implementation in workflow/metrics.go:
func GetPerTaskQueueFamilyScope(
handler metrics.Handler,
namespaceName namespace.Name,
taskQueueFamily string,
config *configs.Config,
tags …metrics.Tag,
) metrics.Handler {
return metrics.GetPerTaskQueueFamilyScope(handler,
namespaceName.String(),
tqid.UnsafeTaskQueueFamily(namespaceName.String(), taskQueueFamily),
config.BreakdownMetricsByTaskQueue(namespaceName.String(), taskQueueFamily, enumspb.TASK_QUEUE_TYPE_WORKFLOW),
tags…,
)
}
Also in latest version v1.27.2. It seems to be similar to 1.19.1 -
metrics.TaskScheduleToStartLatency.With(
metrics.GetPerTaskQueuePartitionTypeScope(
taggedMetrics,
namespaceName,
// passing the root partition all the time as we don't care about partition ID in this metric
tqid.UnsafeTaskQueueFamily(namespaceEntry.ID().String(),
ai.GetTaskQueue()).TaskQueue(enumspb.TASK_QUEUE_TYPE_ACTIVITY).RootPartition(),
shardContext.GetConfig().BreakdownMetricsByTaskQueue(namespaceName,
ai.GetTaskQueue(),
enumspb.TASK_QUEUE_TYPE_ACTIVITY),
),
).Record(scheduleToStartLatency)
Was this a bug in v1.25.2 or purposefully done? Should this be solved by upgrading to v1.27.2?
Thanks.
Suresh