How to ensure that workflows to not run in parallel?

Community Support
1

Hi,

I am using temporal for a deployment tool and I want to ensure that workflows work a specific resource do not run in parallel. So they should be enqueued but only executed when the previous workflows of the same group are completed. I thought I could use the ID to ensure that, but no policy does seem to fit.

I am using Node, just in case it matters.

2

The standard approach is to use the resource ID as workflow ID and SignalWithStart to signal a workflow about new requests, starting workflow if not yet running. Then the workflow can process the signal requests one by one.

3

This helps, but I am not sure about the lifetime of the workflow.

I could do something like this:

export async function deploymentCoordinator({ deploymentId }: { deploymentId: number }) {
  const queue: DeploymentSignal[] = [];
  setHandler(signalDeploymentAction, async (newAction) => {
    queue.push(newAction);
  });

  // Wait until the queue has at least one item
  await condition(() => queue.length > 0);

  while (queue.length > 0) {
    const newAction = queue.shift();
	
  }
}

Then the workflow would only wait us long as there are sigals. But I am not sure if you cannot have raise conditions so thtat signalWithStart does not create a new workflow, but the current one would not receive the signal anymore.

Or I keep the coordinator alive forever, but this is probably also not the best idea.

4

Hi @Sebastian_Stehle

Something like this should work,


export async function deploymentCoordinator({ deploymentId }: { deploymentId: number }, 
                                            queueInput: DeploymentSignal[]) {

  const queue: DeploymentSignal[] = queueInput? queueInput : [];

  setHandler(signalDeploymentAction, async (newAction) => {
    queue.push(newAction);
  });

    // if there is nothing in the queue, we can exit
  while (queue.length > 0) {


    const newAction = queue.shift();

    //start child workflow and wait for it to complete



    // I don't think you will need this but just in case... continue is new if the workflow history grows
    if( queue.length > 0 && wf.allHandlersFinished() && wf.workflowInfo().continueAsNewSuggested){
      return await wf.continueAsNew({
        deploymentId,
        queue
      });
    }


  }
  return;
}

while the workflow has pending signals any client invocation with signalWithStart will send a signal to the open workflow. Once there are no more signals to process you can close the workflow.

I have added the code to CAN in case the workflow history size grows to some reasonable size, but it is probably not needed in your case.

Antonio