Configuring external promethrus and grafana

Community Support
,
3

Hi @whitecrow, are you deploying with docker-compose or helm charts?
If you are using Temporal helm charts with the minimal install:

helm install \
    --set server.replicaCount=1 \
    --set cassandra.config.cluster_size=1 \
    --set prometheus.enabled=false \
    --set grafana.enabled=false \
    --set elasticsearch.enabled=false \
    temporaltest . --timeout 15m

and your cluster is up run:

kubectl get svc
and see all services (temporaltest-frontend-headless, temporaltest-history-headless, temporaltest-matching-headless, temporaltest-worker-headless) that expose port 9090
server metrics are by default enabled for these services on port 9090.

easiest way to view the metrics (and set them as targets in your prometheus scrape config) is just via port-forward, for example:

kubectl port-forward service/temporaltest-matching-headless 9090:9090

and view your matching service metrics at http://localhost:9090/metrics
(you can do this for each individual service for which you want to scrape metrics for)

If you are deploying via docker-compose in your config under temporal->environment enable metrics by setting a prometheus endpoint:

- PROMETHEUS_ENDPOINT=0.0.0.0:8000

and expose port under temporal->ports:

- 8000:8000

and finally view your metrics on http://localhost:8000/metrics
(metrics will be for all temporal services)

if you are deploying your app and worker services via docker-compose as well, see this demo app, especially the “deployment/prometheus” and “deployment/grafana” dirs to see how you can configure scrape points and grafana with dashboards.

hope this gets your started in the right direction