Temporal Server Logs location & Health check context url

jaffarsadik · August 31, 2021, 12:47pm

Hi,

I have deployed the temporal server in kubernetes cluster, I would like to know the how & where to configure the logs for workflow services like frontend, worker, matching & history.

How to check the health check for all the 4 services? is there any default context?

tihomir · August 31, 2021, 7:24pm

Are you looking for server logs or metrics?
In order to get a good picture of your cluster health you should most likely set up metrics, dashboards and possibly alerts.

For health checks see this post for more info.

SDKs in addition perform the gRPC health check when you create a new client.
With tctl you can also health check the front-end service with command:
tctl cluster health

jaffarsadik · September 1, 2021, 1:44pm

@tihomir
Server Logs:
I am looking on server logs of the services frontend, matching, worker & history.
I have deployed each service in separate pods in Kubernetes.
I would like to log some location like /opt/logs,.

Health Check:
Do we have health check url for all the services that can be tested through http(s) url instead of CLI

tctl cluster health

like… https://temporal-server/frontend/healthcheck
https://temporal-server/worker/healthcheck

something like this?

tihomir · September 1, 2021, 4:44pm

For health check you can see sample code here:

github.com

temporalio/temporal/blob/master/tools/cli/clusterCommands.go#L39

    
      
          
          
	"github.com/urfave/cli"
          	healthpb "google.golang.org/grpc/health/grpc_health_v1"
          )
          
          
const (
          	fullWorkflowServiceName = "temporal.api.workflowservice.v1.WorkflowService"
          )
          
          
// HealthCheck check frontend health.
          func HealthCheck(c *cli.Context) {
          	healthClient := cFactory.HealthClient(c)
          	ctx, cancel := newContext(c)
          	defer cancel()
          
          
	request := &healthpb.HealthCheckRequest{
          		Service: fullWorkflowServiceName,
          	}
          	resp, err := healthClient.Check(ctx, request)
          
          
	if err != nil {

Note that fullWorkflowServiceName
should be changed depending on which service you want to health check:

temporal.api.workflowservice.v1.WorkflowService
temporal.api.workflowservice.v1.HistoryService
temporal.api.workflowservice.v1.MatchingService

and as mentioned you can health check FrontEnd service via the tctl cluster health command.

I believe that for server logs it goes to stderr and stdout by default. This should be able to be changed via your deployment infra config tho. Will check more on this and get back to you.

Rob_Temporal · September 1, 2021, 5:02pm

You can configure where the log is written:

alex · September 2, 2021, 1:16am

All services are gRPC endpoints and healthcheck is implemented in standard gRPC way. Please check gRPC doc for more details.
AWS for example, support gRPC healthcheck out of the box.
If you want to do it manually you need a special tool (sort of curl for gRPC) for it. It is very easy to write one in go, or you can use grpcurl or evans.

jaffarsadik · September 14, 2021, 11:58am

@tihomir
Can’t we enable all 4 services healthCheck? How to do it?
is there way we can enable healthCheck for all 1 or 4 services from config deployment file?

Topic		Replies	Views
Connection failure Community Support go-sdk	1	1198	October 15, 2021
Health check working on grpcurl but error on tctl Community Support tctl	1	723	August 12, 2022
Temporal Client/Worker health-check Community Support java-sdk	8	3505	January 21, 2021
Temporal Server Health Check Community Support production	6	6593	January 13, 2024
Temporal history and Matching service health check Community Support tctl , healthcheck	13	1587	March 1, 2022

Temporal Server Logs location & Health check context url

Related topics