Hi everyone,
I’m building a workflow orchestrator on top of Temporal and I’d like to validate whether my current approach aligns with Temporal best practices.
At a high level, I compile a workflow definition into a DAG. From that DAG, I compute an execution order (topological sort). The Temporal workflow then iterates through that order and executes each node using the same activity (ExecuteNode), passing the node ID and a shared execution context.
So conceptually:
-
One Temporal Workflow per execution
-
A DAG representing the workflow topology
-
N nodes in the DAG
-
N executions of the same activity type, each representing a node
For example, if the DAG has 20 nodes, the workflow schedules 20 activity tasks of the same activity type, one per node, sequentially (for now).
My main questions are:
-
Is this a reasonable way to model a DAG-based workflow in Temporal, or am I fighting the framework?
-
Would Temporal recommend a different modeling approach (for example, child workflows per node, or distinct activity types per node)?
-
From an observability and debugging perspective, is having 20 executions of the same activity type a problem? What are the best practices to clearly debug and inspect each node execution (inputs, outputs, failures) in this setup?
-
Are there any scalability or history-size concerns with this pattern as the number of nodes grows?
Below is a simplified version of the workflow code to give more context. The key part is the loop that executes each DAG node via the same activity.
for _, nodeID := range nodes {
var result string
err := workflow.ExecuteActivity(
ctx,
ExecuteNode,
nodeID,
results, // previous node outputs
).Get(ctx, &result)
if err != nil {
return err
}
results[nodeID] = result
}
I’m mostly trying to understand whether this pattern fits well with Temporal’s execution model, and how others would approach debugging and observability for DAG-style orchestrators.
Thanks in advance for any guidance or feedback.