Seeking Guidance: Ensuring High Availability for Continuous Database-Driven Workflows with Temporal

Community Support
1

Hello,

I am currently managing a workflow that is triggered by each row in a specific database table. My requirements for this workflow are as follows:

  1. Infinite Processing: Each row in the table should initiate an infinite activity or a child workflow. This child workflow/activity will, in turn, trigger an infinite activity that takes the row data as input.
  2. Continuous Addition: Should a new row be added to the table, I expect a new child workflow/activity to start independently, without affecting or terminating any existing workflows/activities.
  3. Seamless Redeployment: In the event of a redeployment, it is crucial that a new primary workflow starts in the new container and re-initializes all child workflows/activities based on current table rows. The infinite activities within these child workflows should begin execution in the new environment before the old workflow and its children are terminated. This process should ensure no data loss or interruption in processing.
  4. Error Handling: If any of the child workflows/activities encounters an error, I require an infinite retry mechanism to ensure reliability and continuity of processing.

Temporal has been a fitting solution for our product up until now, and I am seeking guidance to implement these specific requirements effectively. Any assistance or insight on configuring our Temporal workflows to meet these conditions would be greatly appreciated.

1 Like
2
  1. Infinite activity should heartbeat and have a reasonable heartbeat timeout. Infinite workflow should call continue-as-new periodically to keep the history size in check.
  2. You need another workflow to execute a query that would return the newly added rows. Another option is to execute a full scan of the table and call startWorkflow for each row discarding “already started” errors.
  3. Use the “execute a full scan” from (2).
  4. Temporal supports unlimited retries of activities and workflows out of the box.