Long running workflows / activities

Community Support
, , ,
1

Hello, basically i have this workflow to do:
trigger an external system to do some long operations (up to 48H) → monitoring the progress (state changes, speed ecc.) → when it’s done i need to do some ending stuff.

Now, since there could be 1000 instances of the workflow above i have some questions:

  1. best practices for this long running workflow / activies
  2. having 1000 instances of the workflow above means i need 1000 worker to perform that in parallel?

Thanks

2

for activities not to timeout retry you can possibly use AsyncActivityCompletion Sample and do a periodic heart beat to ensure that the activity donot timeout/retried.

As far as workers, i dont think you need 1000 workers for sure , one worker should be able to handle multiple workflows. but how may worker you need exactly is something which depdends on your use case.

3

Hi madhu, thanks for the response, i saw the sample, but my use-case is a little difference.
I don’t need an activity async that when return the workflow keeps going, because i need to perform different stuff once the activity finished, so i don’t want to perform those BEFORE the activity has actually over.
I don’t understand what it means with (in line 101):
“// When doNotCompleteOnReturn() is invoked the return value is ignored.”
The workflow keep going?
The workflow end and the activity keep going?
Or the workflow wait the activity to end? (that’s what i actually need)

Basically the SYNC flow seems to work for me, i NEED to wait that the stuff end (even if it takes 48H) but i need to know the result of that.
Maybe i don’t understand the utility of call an activity in ASYNC mode.

4

doNotCompleteOnReturn indicates that when an activity function returns the activity is not going to complete. So the return value of the function will be ignored as an activity will be completed asynchronously. The main reason for this feature is to support asynchronous implementation of activities to minimize number of threads used by them. If all your activity implementations are synchronous there is no need for this feature.

The 24 hour workflows are not that long when using Temporal. So no special handling is needed. 1000 parallel instances are also pretty low load (some applications start 1+k workflows per second). Temporal scalability is defined not by number of workflows, but how many operations (aka state transitions) happen per second.

1 Like