Can you restart child workflow?

Community Support
1

Hi,

I was wondering if I had a workflow that had multiple child workflow, is it possible to just manually trigger the restart of one of the child workflow or would you have to restart the whole workflow.

Ex. Lets say i have a CI/CD workflow

Trigger --> Build --> IntegrationTest --> Deploy

In this case, each stage is a child workflow of the parent workflow which is CI/CD. Let’s say i just want to retrigger the deploy child workflow. Is that possible.

Thanks,
Derek

2

I filed an issue to add such support through the workflow reset command.

But in your case, I believe the correct solution would be making retriggering a part of the workflow business logic. The parent can cancel and start a child with new data upon receiving a special signal. This way the workflow code has full control over retriggering semantic.

3

Wouldn’t this mean that the parent workflow need to be active the whole time?

Theres two use cases:

  1. Let’s say something went wrong with a workflow and I only want to rerun a specific section of the code, In this case, the workflow is done. Would your suggestion mean that, I would have to start a new workflow with the same parameters, and the parent workflow would have to have enough knowledge to skips certain workflows and jump straight to the desired child workflow?
  2. The other use case, is as stated I want to just trigger redeploy. How would you guarantee the artifacts that are passed from one child workflow to the next with the strategy you proposed? The hope was that since the history for the input for each child workflow was stored, that re-triggering it would replay the exact state vs. a user having to start a new workflow.
4

I was involved in a few CI/CD projects built on top of Temporal. The design we adopted was to have a top level workflow for the whole pipeline that is always up and running and always running children. One such child per pipeline stage. Each stage workflow executes child workflows that perform specific based on some triggering logic. For example a build stage executes a child workflow for each new commit, or performance test stage executes a child workflow every night.

  1. Let’s say something went wrong with a workflow and I only want to rerun a specific section of the code, In this case, the workflow is done. Would your suggestion mean that, I would have to start a new workflow with the same parameters, and the parent workflow would have to have enough knowledge to skips certain workflows and jump straight to the desired child workflow?

If you want to execute some section of the code you’ll have to pass parameters that would allow executing that specific section.

  1. The other use case, is as stated I want to just trigger redeploy. How would you guarantee the artifacts that are passed from one child workflow to the next with the strategy you proposed? The hope was that since the history for the input for each child workflow was stored, that re-triggering it would replay the exact state vs. a user having to start a new workflow.

Children are triggered by a parent. So the parent can store and pass whatever data to any of them.

5

oo interesting…never thought about having a top level workflow that is always running.

  1. (Just thinking out loud) With this approach, that means you have to manage internal state yourself vs. relying on temporal to do it for you. Is there a limit to how much internal state data can be stored? Example if within the workflow, I keep a list that keep track of all the triggered pipelines. Say if there were 1000 of pipelines running concurrently, would this be an issue?
  2. Is there any other recommendations you have for designing a CI/CD pipeline?

Thanks,
Derek

6

I would have an instance of the top level workflow per pipeline. So in this case there will 1000 top level workflows each maintaining a state of a single pipeline.

7

Do you mean that for each unique CI/CD pipeline, you would have a Top level workflow that is always up and running?

I’m probably missing something, but I guess the part i’m not following is if the pipeline is always up, how do you track the whole flow of a pipeline execution that goes through five stages (five child workflow) for example. In the previous strategy where you have a new workflow triggered for each pipeline, you can track the full cycle.

Thanks,
Derek

8

The CI/CD pipelines I had to deal with were more like streaming jobs. They had stages and each stage executed on its own schedule and triggers. It wasn’t just a sequence of steps which is always executed for every commit. For example, a pipeline could have:

  • Build stage which ran verification build and unit tests for each new commit
  • Integration tests that ran once per hour for all commits accumulated during this hour
  • NIghtly performance regression tests that executed for all commits since the last run
  • Release stage that was triggered manually and on average executed once per two weeks.

It is not possible to model such a pipeline as a simple sequence of steps. So we modeled it as always running child workflow per stage with each individual stage run as a child workflow. For the above example, it would have one top-level workflow and 4 children. And each build, test, or release instance would be a child workflow (with the stage being a parent) execution.

Any workflow can be queried for its current state at any time. So visibility is not a problem.

9

In your example, let’s say one CI/CD pipeline consist of triggering the Build stage then the integration test after. You mentioned that each stage had it’s own trigger, but how did the stages talk to each other. Did it rely on the top workflow and request made via top workflow signal method with a config.

Example:
Does the config have a correlation ID and configuration that the top workflow specify the sequence of stages executed and the parameters. If I need to query the all the stages for the execution, would I then just query the correlationID to identify what workflows and the sequence.

10

There are many ways for stages to talk. They can send a signal to the next stage directly by its id. A trigger activity can poll for some artifact or query previous stages for some status. They can rely on timers to execute in many cases.