# Missing workflow data from /metrics

**URL:** <https://community.temporal.io/t/missing-workflow-data-from-metrics/1787>\
**Category:** Community Support\
**Tags:** metrics\
**Created:** [March 22, 2021, 12:53pm UTC](https://community.temporal.io/t/missing-workflow-data-from-metrics/1787 "2021-03-22T12:53:19Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![johanforssell](https://avatars.discourse-cdn.com/v4/letter/j/a87d85/32.png) [@johanforssell](https://community.temporal.io/u/johanforssell)\
**Post date:** [March 22, 2021, 12:53pm UTC](https://community.temporal.io/t/missing-workflow-data-from-metrics/1787/1 "2021-03-22T12:53:19Z")

</div>

Hello

Looking at [https://github.com/temporalio/dashboards/blob/6094dd666f386e76a3c03e0049f02521210b6883/dashboards/sdk.json#L446](https://github.com/temporalio/dashboards/blob/6094dd666f386e76a3c03e0049f02521210b6883/dashboards/sdk.json#L446) I see that there should exist metrics named `temporal_workflow_completed`, `temporal_workflow_failed`, etc.

**I cannot find any of these in the output** from _/metrics_. I’m getting the following values:

> build\_age  
> build\_information  
> client\_errors  
> client\_latency\_bucket  
> client\_latency\_count  
> client\_latency\_sum  
> client\_redirection\_errors  
> client\_redirection\_latency\_bucket  
> client\_redirection\_latency\_count  
> client\_redirection\_latency\_sum  
> client\_redirection\_requests  
> client\_requests  
> event\_blob\_size\_bucket  
> event\_blob\_size\_count  
> event\_blob\_size\_sum  
> gomaxprocs  
> history\_size\_bucket  
> history\_size\_count  
> history\_size\_sum  
> invalid\_task\_queue\_name  
> memory\_allocated  
> memory\_gc\_pause\_ms\_bucket  
> memory\_gc\_pause\_ms\_count  
> memory\_gc\_pause\_ms\_sum  
> memory\_heap  
> memory\_heapidle  
> memory\_heapinuse  
> memory\_num\_gc  
> memory\_stack  
> namespace\_cache\_callbacks\_latency\_bucket  
> namespace\_cache\_callbacks\_latency\_count  
> namespace\_cache\_callbacks\_latency\_sum  
> namespace\_cache\_prepare\_callbacks\_latency\_bucket  
> namespace\_cache\_prepare\_callbacks\_latency\_count  
> namespace\_cache\_prepare\_callbacks\_latency\_sum  
> num\_goroutines  
> persistence\_latency\_bucket  
> persistence\_latency\_count  
> persistence\_latency\_sum  
> persistence\_requests  
> restarts  
> service\_authorization\_latency\_bucket  
> service\_authorization\_latency\_count  
> service\_authorization\_latency\_sum  
> service\_errors\_context\_timeout  
> service\_errors\_entity\_not\_found  
> service\_errors\_execution\_already\_started  
> service\_latency\_bucket  
> service\_latency\_count  
> service\_latency\_sum  
> service\_requests  
> version\_check\_failed  
> version\_check\_latency\_bucket  
> version\_check\_latency\_count  
> version\_check\_latency\_sum  
> version\_check\_request\_failed

Is there something I missed turning on? I’ve set the environment variable `PROMETHEUS_ENDPOINT: "0.0.0.0:9090"` on the frontend node.

I’m using Docker Swarm to run my services. Is there anything from [https://github.com/temporalio/helm-charts/blob/master/values.yaml](https://github.com/temporalio/helm-charts/blob/master/values.yaml) I should set as env to trigger metrics on the workflows?

---

<div class="post-metadata">

**Author:** ![Wenquan\_Xing](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/wenquan_xing/32/362_2.png) [@Wenquan\_Xing](https://community.temporal.io/u/Wenquan_Xing)\
**Post date:** [March 22, 2021, 6:44pm UTC](https://community.temporal.io/t/missing-workflow-data-from-metrics/1787/2 "2021-03-22T18:44:12Z")

</div>

`temporal_workflow_completed` , `temporal_workflow_failed` these metrics are reported by SDK:

> <https://github.com/temporalio/sdk-go/blob/d87457aa1f2fe2dd2ae54ca2cb8f896b7203d4f5/internal/common/metrics/constants.go#L31>

> <https://github.com/temporalio/sdk-java/blob/f0bbf84c496454d9909caefe6ca2784f76b8e592/temporal-sdk/src/main/java/io/temporal/internal/metrics/MetricsType.java#L24>

---

<div class="post-metadata">

**Author:** ![johanforssell](https://avatars.discourse-cdn.com/v4/letter/j/a87d85/32.png) [@johanforssell](https://community.temporal.io/u/johanforssell)\
**Post date:** [March 26, 2021, 1:37pm UTC](https://community.temporal.io/t/missing-workflow-data-from-metrics/1787/3 "2021-03-26T13:37:55Z")

</div>

Thank you for your reply!

Turns out, there’s several issues which I did not understand when I wrote this question.

**First of all** - I was only picking up the metrics from the Temporal FRONTEND node. In order to get a complete set of metric data, one should _also_ set the `PROMETHEUS_ENDPOINT` environment variable for the HISTORY, WORKER and HISTORY nodes.

Will ADMINTOOLS also give you metrics? I don’t know.

**Secondly** - I did not understand that you need to report “your own” metrics directly from your Temporal Client as implemented by the SDK you’re using. There’s the example in [samples-go](https://github.com/temporalio/samples-go/blob/master/metrics/worker/main.go) where we see how to set the MetricsScope when creating the Temporal Client.

```auto
	c, err := client.NewClient(client.Options{
		MetricsScope: newPrometheusScope(prometheus.Configuration{
			ListenAddress: "0.0.0.0:9090",
			TimerType: "histogram",
		}),
	})

```

So - between setting the env correctly for _all_ the Temporal server nodes and reporting from the Temporal Client when using the SDK - I’m now getting all the metrics. I think?

(this post for other people trying to get this working. Is this collected in the documentation in any way, yet?)

---

<div class="post-metadata">

**Author:** ![Wenquan\_Xing](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/wenquan_xing/32/362_2.png) [@Wenquan\_Xing](https://community.temporal.io/u/Wenquan_Xing)\
**Post date:** [March 26, 2021, 8:29pm UTC](https://community.temporal.io/t/missing-workflow-data-from-metrics/1787/4 "2021-03-26T20:29:37Z")

</div>

> [@johanforssell](#):
>
> Will ADMINTOOLS also give you metrics? I don’t know.

admin tools provides the tools for accessing Temporal, I do not expect this pod to emit metrics

---

<div class="post-metadata">

**Author:** ![Wenquan\_Xing](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/wenquan_xing/32/362_2.png) [@Wenquan\_Xing](https://community.temporal.io/u/Wenquan_Xing)\
**Post date:** [March 26, 2021, 8:30pm UTC](https://community.temporal.io/t/missing-workflow-data-from-metrics/1787/5 "2021-03-26T20:30:12Z")

</div>

> [@johanforssell](#):
>
> So - between setting the env correctly for _all_ the Temporal server nodes and reporting from the Temporal Client when using the SDK - I’m now getting all the metrics. I think?

i believe so
