Purpose of Child Workflows?

Community Support
,
1

I am working on a workflow and it is becoming pretty complex and it is becoming hard to read and follow. I was thinking of splitting up a chunk of the work into a child workflow. Is this the point of a child workflow? To help code organization? Or is there something more I am missing.

Thanks!

4 Likes
2

TLDR: There is no reason to use child workflows just for code organization. You can use OO and other code organization techniques to deal with the complexity.

Here are valid reasons to use a child workflow:

  • A child workflow can be hosted by a separate set of workers that don’t contain the parent workflow code. So it would act as a separate service that can be invoked from multiple other workflows.
  • A single workflow has a limited size. For example, it cannot execute 100k activities. Child workflows can be used to partition the problem into smaller chunks. One parent with 1000 children each executing 1000 activities gives 1 million activities executed.
  • A child workflow can be used to manage a resource using its ID to guarantee uniqueness. For example, a workflow that manages host upgrades can have a child workflow per host (hostname being a workflow ID) and use them to ensure that all operations on the host are serialized.
  • A child workflow can be used to execute some periodic logic without blowing up the parent history size. Parent starts a child which executes periodic logic calling continue as new as many times as needed, then completes. From the parent point of view, it is just a single child workflow invocation.

The main limitation of a child workflow versus collocating all the application logic in a single workflow is the lack of a shared state. Parent and child can communicate only through asynchronous signals. But if there is a tight coupling between them it might be simpler to use a single workflow and just rely on a shared object state.

I personally recommend starting from a single workflow implementation if your problem has bounded size in terms of the number of executed activities and processed signals. It is just simpler than multiple asynchronously communicating workflows.

Also, it is frequently overseen that workflows are not just functions, you can use the full power of OO in them. Use structures, interfaces, and other OO techniques to break the logic into more manageable abstractions.

13 Likes
3

Hi Maxim,

How to test the complex workflows without using child workflows? For example, we have a workflow that has 10 steps each of which calls various activities. Every step is currently implemented as a separate method of a workflow class but it is a nightmare to test such a workflow by calling the “main” workflow method mocking every activity call instead of calling “step1()”, “step2()” methods in tests directly (Temporal internals throws an exception when I tried that).

4

Write your own “unit test” workflow that invokes any part of the workflow under test.

5

Maxim,

Do you have an example? I tried many things and they all failed:

  1. I can’t create an instance of the main workflow class (MyWorkflowImpl) form the test workflow (MyWorkflowForTestingImpl) via “new MyWorkflowImpl()” - I get “java.lang.IllegalStateException: Null workflowId. Was workflow started?”

  2. I tried creating a stub for it as a sub-workflow but then I can’t expose the methods I need to test since there could be only one @WorkflowMethod

  3. I tried annotating these methods with @QueryMethod but that also fails with “java.lang.IllegalStateException: Null workflowId. Was workflow started?” since query methods can only be called when workflow is in progress.

Here is my code I use for testing:

@WorkflowInterface
public interface MyWorkflow {
    @WorkflowMethod
    int mainMethod(int param);

    @QueryMethod
    int step1(int param);

    @QueryMethod
    int step2(int param);
}

public class MyWorkflowImpl implements MyWorkflow {

    private final ActivityOptions activityOptions = ActivityOptions.newBuilder()
            .setScheduleToCloseTimeout(Duration.ofMinutes(5))
            .setRetryOptions(RetryOptions.newBuilder().setMaximumAttempts(1).build())
            .build();

    private final MyActivities myActivities = Workflow.newActivityStub(MyActivities.class, activityOptions);

    @Override
    public int mainMethod(int param) {
        var result1 = step1(param);
        return step2(result1);
    }

    @Override
    public int step1(int param) {
        return myActivities.increment(param);
    }

    @Override
    public int step2(int param) {
        return myActivities.decrement(param);
    }
}

public class MyActivitiesImpl implements MyActivities {
    @Override
    public int increment(int param) {
        return param + 1;
    }

    @Override
    public int decrement(int param) {
        return param - 1;
    }
}

TESTS:

@WorkflowInterface
public interface MyWorkflowForTesting {
    @WorkflowMethod
    int mainMethodTest(int param);

    @QueryMethod
    int step1Test(int param);

    @QueryMethod
    int step2Test(int param);
}

public class MyWorkflowForTestingImpl implements MyWorkflowForTesting {

    private static final MyWorkflow myWorkflow = Workflow.newChildWorkflowStub(MyWorkflow.class);

    @Override
    public int mainMethodTest(int param) {
        return myWorkflow.mainMethod(param);
    }

    @Override
    public int step1Test(int param1) {
        return myWorkflow.step1(param1);
    }

    @Override
    public int step2Test(int param2) {
        return myWorkflow.step2(param2);
    }
}

class MyWorkflowImplTest {
    private static final MyActivities myActivities = new MyActivitiesImpl();

    @RegisterExtension
    public static final TestWorkflowExtension testWorkflowExtension =
            TestWorkflowExtension.newBuilder()
                    .setWorkflowClientOptions(TemporalConfiguration.getWorkflowClientOptions("default"))
                    .setWorkflowTypes(MyWorkflowForTestingImpl.class, MyWorkflowImpl.class)
                    .setActivityImplementations(myActivities)
                    .build();

    @Test
    void main(MyWorkflowForTesting workflow) {
        var res = workflow.mainMethodTest(5);
        Assertions.assertEquals(5, res);
    }

    @Test
    void step1(MyWorkflowForTesting workflow) {
        var res = workflow.step1Test(5);
        Assertions.assertEquals(6, res);
    }

    @Test
    void step2(MyWorkflowForTesting workflow) {
        var res = workflow.step2Test(5);
        Assertions.assertEquals(4, res);
    }
}
6

Don’t use child workflows. Invoke your workflow methods directly. Note that you will have a separate workflow implementation type per test.

7

Thanks! It worked with a small modification:

private final MyWorkflowImpl myWorkflow = new MyWorkflowImpl();

since MyWorkflow interface can’t have these step1 and step2 methods so I have to use implementation class.

I also used one testing workflow class that implements multiple interfaces:

@WorkflowInterface
public interface MyWorkflowForTestingStep1 {
    @WorkflowMethod
    int step1(int param);
}

@WorkflowInterface
public interface MyWorkflowForTestingStep2 {
    @WorkflowMethod
    int step2(int param);
}

public class MyWorkflowForTesting implements MyWorkflowForTestingStep1, MyWorkflowForTestingStep2 {

    private final MyWorkflowImpl myWorkflow = new MyWorkflowImpl();

    @Override
    public int step1(int param1) {
        return myWorkflow.step1(param1);
    }

    @Override
    public int step2(int param2) {
        return myWorkflow.step2(param2);
    }
}
8

Maxim, as the size increases, will there be a performance hit?
How quickly does the performance degrade? Is it linear or exponential?

9

Maxim, as the size increases, will there be a performance hit?

The performance hit is only for recovering workflow executions after they were pushed out of worker cache.

How quickly does the performance degrade? Is it linear or exponential?

It depends on the workflow. It is linear in the majority of practical cases.

10

BTW @Dominik has the opposite recommendation on this.

11

Maxim is the authority here: He was already doing workflows while I was still doing databases and queues Haha.

Seriously though: Workflows are a great unit of organization or functional decomposition on a service level. Let’s take a Payment Workflow as an example: The Payment Workflow may be developed and operated by its own team and may be invoked from many different workflows in the system that are themselves developed and operated by their own teams.

Here, calling the Payment Workflow as a child workflow makes perfect sense

Now, should you further decompose the Payment Workflow into multiple other workflows?

Here, I would go with Max’s recommendation of “I personally recommend starting from a single workflow” and only stray from that path if you run into specific reasons

tl;dr

Rule of thumb

:+1: Parent/Child workflow inter service
:-1: Parent/Child workflow intra service

1 Like
12

I found this topic because I share the “becoming hard to read and follow” concern as OP but don’t really see that part clearly addressed. I have a workflow that queries for a bunch of record ids, and then iterates over them, executing a handful of activities for each. But in the Temporal UI the result is all the activities are mixed in with each other in the history and so following the process for a single record (i.e. trying to troubleshoot what happened to it) is not easy. Is this a case where I should be spawning the group of activities into its own workflow? It is still an intra-service scenario. Is there a better way to do this kind of filtering via the UI?

1 Like