Connection retry for temporal

Community Support
1

Hi i have installed temporal in my linux system and i use the temporal server start-dev command to run a development cluster
My application is in nestjs and i have written a provider file to connect to this cluster
The code for the file is given as below :

export const TemporalClientProvider: Provider = {
  provide: TEMPORAL_CLIENT,
  inject: [LoggerService, ConfigService],
  async useFactory(logger: LoggerService, configService: ConfigService) {
    try {
      const connection = await Connection.connect({
        address: `${configService.get('TEMPORAL_CLUSTER_IP_ADDRESS')}:${
          configService.get('TEMPORAL_CLUSTER_PORT') || '7233'
        }`,
      });
      return new WorkflowClient({
        connection,
        namespace: configService.get('TEMPORAL_CLUSTER_NAMESPACE'),
      });
    } catch (err) {
      logger.error(`Failed to connect to the temporal server. Error - ${err}`);
    }
  },
};

Here the namespace is given as default, now when i suddenly exit from my temporal in the terminal, i keep getting a temporal connection refused log again and again given as below

2024-02-13T06:45:05.430569Z  WARN temporal_client::retry: gRPC call poll_workflow_task_queue retried 8 times error=Status { code: Unavailable, message: "error trying to connect: tcp connect error: Connection refused (os error 111)", source: Some(tonic::transport::Error(Transport, hyper::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })))) }
2024-02-13T06:45:05.529139Z  WARN temporal_client::retry: gRPC call poll_workflow_task_queue retried 8 times error=Status { code: Unavailable, message: "error trying to connect: tcp connect error: Connection refused (os error 111)", source: Some(tonic::transport::Error(Transport, hyper::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })))) }
2024-02-13T06:45:05.760595Z  WARN temporal_client::retry: gRPC call poll_workflow_task_queue retried 8 times error=Status { code: Unavailable, message: "error trying to connect: tcp connect error: Connection refused (os error 111)", source: Some(tonic::transport::Error(Transport, hyper::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })))) }
2024-02-13T06:45:05.954094Z  WARN temporal_client::retry: gRPC call poll_activity_task_queue retried 8 times error=Status { code: Unavailable, message: "error trying to connect: tcp connect error: Connection refused (os error 111)", source: Some(tonic::transport::Error(Transport, hyper::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })))) }
2024-02-13T06:45:06.179704Z  WARN temporal_client::retry: gRPC call poll_activity_task_queue retried 8 times error=Status { code: Unavailable, message: "error trying to connect: tcp connect error: Connection refused (os error 111)", source: Some(tonic::transport::Error(Transport, hyper::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })))) }
2024-02-13T06:45:06.337474Z  WARN temporal_client::retry: gRPC call poll_activity_task_queue retried 8 times error=Status { code: Unavailable, message: "error trying to connect: tcp connect error: Connection refused (os error 111)", source: Some(tonic::transport::Error(Transport, hyper::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })))) }
2024-02-13T06:45:06.508890Z  WARN temporal_client::retry: gRPC call poll_workflow_task_queue retried 8 times error=Status { code: Unavailable, message: "error trying to connect: tcp connect error: Connection refused (os error 111)", source: Some(tonic::transport::Error(Transport, hyper::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })))) }

I can write a process.exit(1) in the catch of the provider to exit from this, but what i want is i want to have a retry mechanism where after a set number of attempts, it gets exited and also we should start to connect again if we run the app again

Are there any such available retry techniques for typescript?

Thanks and Regards,
Rahul

2

You should catch the Worker.run promise rejection and handle that appropriately.

If you want the application to quit, that’s where you’ll want to exit.

I noticed the nextjs plugin doesn’t handle this properly, you may need to address it there.

You’ll want the application to wait until run resolves before exiting the process for graceful shutdown.