We’re building a system where users can define custom workflows (DAGs) using our features and Temporal — e.g., “create record,” “update record,” etc. So the workflow/activity definition is very dynamic.
Our current design is:
-
A Temporal orchestration workflow starts when triggered or scheduled.
-
It runs an orchestration activity that fetches the workflow definition from our DB (list of steps represented in DAG).
-
By looping through each step using for loop, it determines which predefined Temporal workflow to call (via Java switch cases) inside that activity.
-
Each of those workflows then calls its corresponding activity.
However, this feels inefficient since each step spawns a workflow that runs only one activity, and I feel like I’m calling way too many workflows? Should we replace these step-specific workflows with dynamic activities/workflows instead? Or is this architecture reasonable as is?
Alternatives considered:
Alternative A: I just only call one dynamic workflow after fetching the steps, then there, using for loop, loops through each step and calls dynamic activities. (I’m not sure dynamic workflows/activities are scalable?)
Alternative B: Don’t call another workflow from the orchestration activity. Instead, call relevant services from the orchestration activity. (But I’m worried about this activity getting too big as we introduce more features)
Any best practices/design recommendation is welcome and thank you in advance!