[ architecture ] async child workflows and signals

Community Support
1

Hello, just getting started with Temporal and trying to re-architect my microservices into workflows. :sunglasses:

I’m trying to understand the best pattern to trigger async child workflows that return a value to the parent workflow which needs to collate the net result and make a decision.
Each child workflow is independent to others so should be run in parallel.
As each analysis takes about 3seconds to complete.
And there may be 20 analysis to run on 1 dataset.
The net result need to be calculated as fast as possible.

EDIT: after some more research this post on SO by maxim seems to be the idea SO post

Some pseudo-code to explain more clearly hopefully.

workflow getNetResult ( dataSet1 ) {
   activeAnalysis = [ ]
    activeAnalysis = getActiveAnalysisForDataSet( dataSet1 )
    // activeAnalysis =  [ analysis1, analysis2 ... ]

    analysisResults =  [ ]

    for each activeAnalysis {
        // the below should all be run async at same time
       // rather than waiting for each one to complete
        analysisResult = workflow.run( analysis1, dataSet1 )
    }

    // wait for a signal from child workflows
    // some may complete fast other slower
    workflow.signalChannel(signalChannelName).Receive( analysisValue )
    // analysisValue = +2 or -1 etc..
    analysisResults.push( analysisValue ) 

    // some check to see if all the workflows have triggered?
    // probably by removing the analysis name from 
    // the activeAnalysis array until empty

    // wait for all analysis workflows to complete
    netResult = sumEachElement( analysisResults )

    saveToDB(netResult)
}

Any advice would be much appreciated :+1:
Cheers

2

I’m not sure why you need signals here. ExecuteChildWorkflow in Go returns a Future, so you can start all the child workflows in parallel and then wait for all them to complete using a simple loop that calls Get on each Future.

2 Likes
3

Thanks for getting back to me Maxim!

Checking out some of the samples on Github, which are very useful, looks like I would create a selector, add the future returned from ExecuteChildWorkflow to the selector and check for errors, then for-loop on the select to wait for result?

var result someType
selector := workflow.NewSelector(ctx)
	for _, s := range analysisToRun {
		future := ExecuteChildWorkflow(s)
		selector.AddFuture(future, func (future){
                future.Get(ctx, &result) //get result of analysis here
                /// ... some error handling
        })
        
	}

    // this will wait for all analysis to run
	for i := 0; i < len(analysisToRun); i++ {
		selector.Select(ctx) // this will wait for one branch
        /// ... some error handling
	}
4

If you want to use the results as they become available yes. You could also just store the futures in an array and call .Get on those if the order of completion is irrelevant.