Hi,
I’m running into an issue setting up a datadog dashboard for temporal due to the way the metric names are formatted, for example this is a metric produced now by the frontend service:
temporal.service_latency.namespace.default.operation.DescribeNamespace.service.temporal.type.frontend.95percentile
It looks like tags are being appended to the metric name with dot delimited notation. I’d expect namespace, operation, service, and type to be tags on this metric with the metric name being “temporal.service_latency”.
The reason this metric naming scheme makes it difficult to create datadog dashboard is that each individual combination above needs to be manually added as a series.
Our server config (for the matching server), is as follows:
global:
metrics:
tags:
service: temporal
type: matching
statsd:
hostPort: localhost:8125
prefix: temporal
My apologies if this has been covered elsewhere, I couldn’t find anything similar to this post. I’ve seen some discussion around using Prometheus, but it seems like using the statsd config should just work in this case. I’ve also seen some discussion around using a custom reporter which may be what we want here?
Thank you!