How to retry to connect to workflow client and workers

Community Support Server Deployment
,
1

Hi
i have a nestjs application and i use temporal for creating workflows in certain use cases and i use providers to connect to the temporal client and also to create workers as such

export const TemporalClientProvider: Provider = {
  provide: TEMPORAL_CLIENT,
  inject: [LoggerService, ConfigService],
  async useFactory(logger: LoggerService, configService: ConfigService) {
    try {
      const connection = await Connection.connect({
        address: `${configService.get('TEMPORAL_CLUSTER_IP_ADDRESS')}:${
          configService.get('TEMPORAL_CLUSTER_PORT') || '7233'
        }`,
      });
      return new WorkflowClient({
        connection,
        namespace: configService.get('TEMPORAL_CLUSTER_NAMESPACE'),
      });
    } catch (err) {
      logger.error(`Failed to connect to the temporal server. Error - ${err}`);
    }
  },
};

My worker is also created using a provider as such

export const ReminderWorkerProvider: Provider = {
  provide: TEMPORAL_WORKER_REMINDER,
  inject: [
    LoggerService,
    CollaboratorService,
    ConfigService,
    ContractService,
    OrganisationService,
    UserService,
    ContractDocumentService,
    UserMailService,
    EventService,
    TaskService,
    ContractNotificationService,
    TaskNotificationService,
    EventUserService,
  ],

  async useFactory(
    logger: LoggerService,
    collaborateService: CollaboratorService,
    configService: ConfigService,
    contractService: ContractService,
    organisationService: OrganisationService,
    userService: UserService,
    contractDocumentService: ContractDocumentService,
    userMailService: UserMailService,
    eventService: EventService,
    taskService: TaskService,
    contractNotificationService: ContractNotificationService,
    taskNotificationService: TaskNotificationService,
    eventUserService: EventUserService,
  ) {
    try {
      const connection = await createTemporalWorkerConnection(configService);
      const worker = await Worker.create({
        workflowsPath: require.resolve(__dirname + '/workflow'),
        activities: reminderWorkflowActivities(
          collaborateService,
          contractService,
          organisationService,
          userService,
          contractDocumentService,
          userMailService,
          eventService,
          taskService,
          contractNotificationService,
          taskNotificationService,
          eventUserService,
        ),
        taskQueue: REMINDER_TASK_QUEUE,
        namespace: configService.get('TEMPORAL_CLUSTER_NAMESPACE'),
        connection,
      });
      worker.run();
    } catch (error) {
      logger.error(`Failed to initiate the temporal worker - ${error}`);
    }
  },
};

In certain cases while i am starting my application using npm run start, the temporal cluster might not be active so i usually get this error

error: [2024-02-15T11:11:24.632Z][undefined] Failed to initiate the temporal worker - TransportError: tonic::transport::Error(Transport, hyper::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })))
error: [2024-02-15T11:11:42.460Z][undefined] Failed to connect to the temporal server. Error - Error: Failed to connect before the deadline

after this error comes up, my application successfully runs and listens on its port, but when i then start my cluster, my application does not try to connect to it unless i restart my application so is there a way to know when the cluster is created and then retry the temporal client and worker connection based on that?

Thanks and Regards,
Rahul

2

If you want to start your app and server concurrently, you should retry connecting to the server in a loop for a set duration or number of attempts with a short delay between each iteration.