Temporal Querying Problem

Community Support
1

In Temporal, for a single workflow ID, there can be multiple executions over time.

Right now, when we query workflows, we get all executions for each workflow ID.

However, the requirement is to get only the latest execution for each workflow ID (based on the most recent start time), whether it is running or completed.

The issue is that Temporal does not provide a direct way to filter and return only the latest execution per workflow ID.

Workflow ID Run ID Start Time Status
A r1 10:00 AM Completed
A r2 10:05 AM Failed
A r3 10:10 AM Running
B r4 10:02 AM Completed
B r5 10:08 AM Running

:backhand_index_pointing_right: Current Output (what we get)

All executions:

  • A → r1, r2, r3

  • B → r4, r5

:backhand_index_pointing_right: Expected Output (what we want)

Only latest per workflow:

  • A → r3 (latest)

  • B → r5 (latest)

Why this is a problem

  • Since each workflow ID can have multiple executions, the total number of records becomes very large.

  • When we query this data, it takes more time because we are fetching unnecessary older executions as well.

  • After fetching, we still need to manually filter (deduplicate) the data on the client side to keep only the latest execution per workflow ID.

  • This makes the whole process:

    • inefficient

    • slower

    • and extra work on the client side

2

Hi,

The issue is that Temporal does not provide a direct way to filter and return only the latest execution per workflow ID.
This is a known limitation, ref List Workflow Executions ignoring previous runs · Issue #351 · temporalio/temporal · GitHub

For complex queries, the recommendation is to push the data to an external database. There are several strategies, it depends on your requirements

  • query periodically (listWorkflowExecutions) and update the external source
  • Add an activity at the end of your workflow execution to push the data
  • observe the internal workflow state and run an activity if it changes

Also note that the visibility API (ListWorkflowExecution) is eventually consistent

Antonio

3

Have you looked at the DescribeWorkflowExecution call (temporal workflow describe)? It only requires workflow ID, and will give you back the latest run ID for that workflow ID.