Grpc health probe failed

Community Support
,
1

I try to use grpc-health-probe to do a health check, Instead of the original tcp check. I would like to check three services, include frontend, history and matching, and my ‘exec Liveness probe’ is respectively:

  1. grpc-health-probe -addr=:7233 -service=temporal.api.workflowservice.v1.WorkflowService
  2. grpc-health-probe -addr=:7234 -service=temporal.api.workflowservice.v1.HistoryService
  3. grpc-health-probe -addr=:7235 -service=temporal.api.workflowservice.v1.MatchingService

But the liveness probe failed after I install temporal by helm-chart, however I go into the pod and manually execute the above command, I can get the result “status: SERVING”, which indicates the grpc is normal. So I really don’t know what’s wrong, and how to fix it?

1 Like
3

Are you using charts from this repo?
If so, can you show result of

kubectl get svc

Do your frontend, history, matching services have external ip? With Temporal helm charts all these services have ClusterIP type which makes them only accessible within the cluster.

4

If you are running locally via helm charts you could change services to NodePort type here

and if you dont have external cluster / nodeport ip, can port-forward for example history service:

kubectl port-forward services/temporaltest-history-headless 7234:7234

and can run grpc-health-probe -addr=localhost:7234 -service=temporal.api.workflowservice.v1.HistoryService outside of cluster, if that helps

5

My description may be misleading you. I have rebuilded the tempooral-server image and copy the grpc-health-probe into image, so the grpc-health-probe can access the port within the cluster, and I use it as ‘Exec Liveness Probe’.