Is it possible to dynamically add a child workflow to an already created cancellationScope?

Community Support
1

Hello,

To provide some context on my use case, I want to build a file delivery workflow pretty similar to the file processing workflow from the official github sample page file processing JDK sample

Lets call this workflow “Sending Workflow”. Basically each sending workflow will deposit processed files in a folder on a server (one folder for each Sending workflow)

The difference is that I also have these specific requirements:

  1. Delivery workflow has to be grouped in batches:
    • Batch 1 (Folder /batch1):
      • File of sending Workflow 1 are inside Folder /batch1/workflow1
      • ….
      • File of Sending Workflow N inside Folder /batch1/workflowN
    • Same for Batch 2
  2. Only after the condition for my batch to be closed (either after X minutes or after X number of delivery), i have to notify the receiver that he can ingest the newly created batches.
  3. I Should have the ability to dynamically add sending workflow to a batch because I don’t have the number of Sending workflow in advance. These workflow are triggered by an event and are therefore coming continuously
  4. If any workflow remaining in the batch is yet to be completed when the condition is reached, I should cancel these workflow.

For this use case, I would like to know if it is possible to dynamically trigger a child workflow as part of an already initialized cancellation scope using a signal ? My idea is to generate a parent batch workflow with one child sending workflow using a cancellationScope. From there trigger the following child workflow as part of the cancellationScope using signals.

If it is not possible would the same approach but without relying on cancellation scope be appropriate? The flow would then be to trigger async child workflow using a signal to the parent (no cancellationScope) and if a cancellation condition is reached, send multiple signals to all running child workflow to trigger a clean cancel.

Thank you again for your help.

2

Don’t close the scope. Something like:

 CancellationScope scope =
      Workflow.newCancellationScope(
          () -> {
            startChildWorkflow(new ChildInfoImpl());
            while(!isCancelled) {
               ChildInfo childInfo = getNextChildFromQueuePopulatedBySignalHandler();
               startChildWorkflow(childInfo);
            }
          });
  scope.run();
3

Thanks for the reply maxim.

The solution provided works well if I want to wait to have X number of child workflow before starting each child workflow processing at the same time. However in my process, i need my childWorkflow to start their processing without waiting to have X number of child workflow “subscribed” to my parent workflow. Using a list of CancellationScope doesn’t seem like a right approach neither as my batch can have up to 50 child workflows (number of batch can easily increase from there).

What I ended doing for the moment is to not use cancellation scope when initializing my child workflow in async mode:

  • I initiate a batch parent workflow with one initial child workflow. The parent workflow has an activity which is continuoulsy checking for the closing condition. Which means that even if the child workflow completes, it is valid for my batch to still be open waiting for additional child workflow
  • I can subscribe a new child using a signal to the parent workflow
  • If the condition for the batch to close is reached, I send a signal to all remaining child workflow which haven’t completed yet to cancel them and wait for all of them to complete their cleanup process
  • I finally complete my parent batch workflow with the last activity to notify my receiver

This kinda works but the downside is that my cancelled child workflow are not in a cancel state anymore but complete (after performing the cleaning of activity). Throwing back the activity error from my child does not work as the child workflow has been initialized without cancellation scope (would result in failure). Is there a way using the SDK to set my child workflow in cancel state from within itself (without an external client) after it has performed its cleaning activity?

Thanks again for your time.

4

The waiting is unrelated to CancellationScope. So you can start them in parallel and invoke new child workflows when needed in the same scope as I outlined.

5

Hello Maxim,

Just reviewed the code to fit the solution proposed and it works perfectly. Don’t know why i though the child wouldn’t be started correctly.

Thanks again for your help.