Modelling workflows with concurrent activities

Community Support
, ,
1

Hi,

I’m modelling an email subscription workflow (using the Go SDK), where a user can be matched to 1 or more third party providers.
When there’s > 1 match I’d like to start the activities concurrently, as I don’t mind which order they are done in, just that they complete.

I can see there’s workflow.Go can/should I:

  1. Use workflow.Go with a stdlib Waitgroup to ensure my workflow blocks until N activities completes?
  2. Start each activity and just return from the workflow then, if so, what does that mean for completion?
  3. Convert each activity to a workflow, start them as child workflows but return immediately?

I’m concerned that some providers are pretty unreliable and as such, I’d rather not delay the API calls to the reliable ones (and I’d prefer not to maintain an order of priority for providers if I can help it).

Thanks

2

There is no need to use workflow.Go to execute an activity in parallel. ExecuteActivity call is already asynchronous returning a Future. See branchworkflow sample.

3

Maxim, perfect thanks.

1 Like
4

Are there any best practices around collecting & returning multiple errors?

Using the branch workflow as an example, the first future to return an error causing the workflow to return, what if you have > 1 future that errors?

In normal concurrent Go code, I’d collect the errors and return them as a multi-error (or concatenated), is that the recommended approach?

// accumulate results
var errs []error
for _, future := range futures {
	if err := future.Get(ctx, nil); err != nil {
		errs = append(errs, err)
	}
}

// If we have any errors, combine into a single error and return
if len(err) > 0 {
	return MutilError(errs...)
}

Thanks.

5

The best practice is to not fail workflow on intermittent errors. Note that in the workflow world an intermittent error can be very long running in some cases. For example a downstream dependency be down for a few days.

So ideally you would set up appropriate retry options that would retry activities until they complete instead of failing the workflow.

6

That makes sense, and highlights I need to consider workflow error vs expected failure case better, I agree that the activity should always be completable.