SDK metrics not showing up

Community Support
,
1

Hi, I am trying to get out-of-the-box java SDK metrics, ran the docker image from here. Follow the code example from here.From grafana, I can see server metrics, but sdk metrics are not showing after I ran a few jobs.

I do see the sdk scraping endpoints are down, may be this is related? Can someone give me a pointer on how to get this running?

image
image904×911 109 KB

3

So on the docker compose side the two ports that are targeting scraping of SDK metrics are 8077 and 8078. Also the metrics path is set to /metrics.

To get the sample to work with this setup:

  1. Change client port to 8077
  2. Change worker port to 8078
  3. Change metrics endpoint to /metrics

run the sample and check your prom targets again they should be green (and then check the grafana sdk dashboard)

Would gladly take a PR to the sample with these changes if you feel like contributing. Thanks!

1 Like
5

Thanks @tihomir , the reason I was trying to get the settings up is to verify the tagging of SDK metrics, we have a set up with reporter set to a local statsd (running in a side car) and those got push to DataDog. That seems to work ok, except all the SDK metrics showing up in DD do not have activity_type nor workflow_type tagged. I wonder if there is anything I missed, or does Temporal support this setup?

statsDClient = new NonBlockingStatsDClientBuilder()
        .prefix("myprefix")
        .hostname(statsdHost)
        .port(8125)
        .build();
Scope scope =
        new RootScopeBuilder()
            .reporter(new StatsdReporter(statsDClient))
            .reportEvery(Duration.ofSeconds(10));
WorkflowServiceStubsOptions.newBuilder()
            .setMetricsScope(scope)
            .setTarget(host + ":" + port)
            .build());
6

I resolve this by using Spring Boot actuator metricRegistry, and tags are showing up properly. Thanks.