How to receive intermediate results while the workflow is still running?

Community Support
1

I have a media-processing workflow with multiple activities. The first activity downloads a file, then all following activities process that file in parallel (using the “worker-specific task queues” pattern). When any of the activities completes, I’d like to receive the result of that activity immediately while other activities continue.

I’m considering two approaches:

  1. The client polls repeatedly using queries
  2. The client exposes a HTTP endpoint as callback, and the workflow post activity results to that endpoint

Polling is obviously not ideal. Using a HTTP endpoint kinda works, but is quite complex to implement. I wonder if there is a better way?

2

You could use Update to wait for workflow to reach some state.

Here is a more relevant sample in Java:

1 Like
3

Follow up question: can I trigger an update after the workflow has completed?

I understand that queries can be used on completed workflow, but it would be nice if the caller can use the same API to retrive results regardless of whether the workflow has completed or not.

As an aside, is there a technical reason why updates cannot operate on completed workflows like queries?

4

Workflow updates are recorded in event history, event history for completed executions is immutable. Workflow queries do not add to event history.
Also you can call Temporal apis like schedule activities inside update handlers which would also not work for completed executions. This is not allowed inside query handlers.

1 Like