Hi there, I’m currently evaluating temporal to be used in our org. I’m super excited that I found this, seems to be a great fit for us.
I’m still starting reading into the docs, and I must confess I have not yet deployed a sample app.
But looking into the concepts I’m trying to map it to our needs, and one of the main use cases we have is the capability to chunk requests into thousands (so far our max is 7k) parallel functions that once completed they need to be assembled (something like a map/reduce execution model).
So I have a few questions:
From the workflow/child docs, it seems that there’s a max number of activities a workflow can track. What is that number? 1000?
Each function is executed remotely via message passing into a kafka cluster, the nodes on the other end may take from 2-60 minutes to send a reply, looking at the examples is Async.function the way to go here? I’m assuming I could have a loop 1…N chunks that creates the Promise for each function and then invoke allOf to get the results of each execution. In this scenario, how do I control individual errors/retries? What if one of the chunks fail and the workflow gets stuck on a never ready state?
Besides those basic questions, one thing I’m still not clear on, is the design of the workflow. I’m used to workflows such as activity for instance, to be composed of several smaller steps. It looks to me (and I’m assuming that I can just not getting it yet) that a @WorkflowMethod is a single point of entry/end of a workflow. But if I have dozens of activities within this workflow, it was not clear to me how to orchestrate them. For instance how the uploadActivity once done can continue into the split activity that once done will continue to the mergeActitivy and finally the notificationActivity? Could someone care to give an pseudo example?