I’m running into an issue setting up a datadog dashboard for temporal due to the way the metric names are formatted, for example this is a metric produced now by the frontend service:
It looks like tags are being appended to the metric name with dot delimited notation. I’d expect namespace, operation, service, and type to be tags on this metric with the metric name being “temporal.service_latency”.
The reason this metric naming scheme makes it difficult to create datadog dashboard is that each individual combination above needs to be manually added as a series.
Our server config (for the matching server), is as follows:
global: metrics: tags: service: temporal type: matching statsd: hostPort: localhost:8125 prefix: temporal
My apologies if this has been covered elsewhere, I couldn’t find anything similar to this post. I’ve seen some discussion around using Prometheus, but it seems like using the statsd config should just work in this case. I’ve also seen some discussion around using a custom reporter which may be what we want here?