Question about process tree-like workflows

Community Support
1

Hi,

I wanted to get some advice on what the best way to design my current workflow. I have a tree like workflow where each node is a child-workflow and that child node can have child-workflow. You can imagine that each node has information about the current node as well a list of children nodes.

{
   name: string
   children: Queue<Node>
}

The tricky part is that you don’t have a view of the whole tree and you only have the view of the top layer. As you process each node via activity, the activity will signal back what child nodes are left. These nodes will be added to the workflow via signal.

So right now, this can be done if I block until the child workflow is complete to process the next childworkflow. However, I wanted to know is there a way where I could process it all async.

Example:

CiCdWorkflow --> BuildWorkflow --> Job1 workflow
–> Job2 workflow
–> Job 3 workflow

Step 1) CiCdWorkflow starts the buildworkflow with a list of Jobs(Job1, Job2, Job3). The buildworkflow puts the jobs into a queue and does a loop until queue job is empty.
Step 2) let’s say Job1 executes. It completes via signal to the job workflow. The signal will also contain a list of childnodes to process next. The job1 workflow will complete and then return with a list of children node. BuildWorkflow will add it to the queue of jobs.

Because in step 2, I depend on the return on the children nodes from the job workflow, it’s not clear to me how you would achieve this with a promise since there seems to be only two options:

  • Promise.all which waits for all the jobs at the level to be complete before starting children. Since each job can finish at different rates I want each node to start the children nodes when it’s complete.
  • Promise.anyOf this won’t work because I need the return of the child nodes from the job workflow.

Any suggestion appreciated.
Thanks!

2

Sorry, I think I still don’t understand your problem. Do you need to unblock workflow code on any of the child workflows completing as well as on receiving a signal? In this case, you can use Promise.anyOf and include an additional promise that will be completed by a signal handler.

3

Here is an example
Job1 --> Job 1-a --> Job 1-a-c
Job2 --> Job 2-a --> Job 2-a-c

Step 1:

  • Execute job1 and job2 by creating a promise.

If i do Promise.all(job1, job2), both job1 and job 2 need to be completed at the same time. So that won’t work

If i do Promise.anyOf(job1, job2), then it will return if either job1 or job 2 finish. Job1 and Job2 will return the child nodes once its complete executing. In the case the job1 finished before job 2, I have the child nodes of job1, but I don’t have it for job2. I need the child nodes of job2 to process it as well. Maybe i’m missing something on how to handle that case with promises.

4

I still don’t fully understand the requirements. Can you call anyOf on the list of Promises and update that list every time any of the children or their subjobs complete?

5

I guess maybe i’m not understanding how you would do that with Promise.anyof

List children = Promise.anyOf(promise1, promise2, promise3);

If promise1 finishes first, then you get then the promise 1 children. However at this point, how would you get the return of promise2 and promise 3?

6

Something like:

Promise.anyOf(promise1, promise2, promise3).get();
if (promise1.isReady()) {
    children.addAll(promise1.get());
}
if (pormise2.isReady()) {
   children.addAll(promise2.get());
}
if (pormise3.isReady()) {
   children.addAll(promise3.get());
}

Then you can use Promise.anyOf that waits on a list and execute this logic in a loop if needed.

7

Thanks! sorry i didn’t realize the promise had that API.