Drop pending workflows

Community Support
1

I have a remote worker that is doing some processing and is called from a workflow which only invokes an activity on the remote worker.

This workflow could be invoked multiple time in a short period of time, while the worker takes longer time to process each request. However, each call to the worker is an of an updated state, so this means that only the last request is important.
Each workflow could have a unique ID based on the resource I wish to update, so it is easy to identify when if I have multiple workflows (all of which but one are redundant) which ones I need to drop.

Is there a way to manage the queue of pending workflows so that I would be able to prevent the worker from processing ALL requests, and instead only run the last request available?

For example, say I have a camera service taking a single image per 1-10 second, and sending this image to a processing workflow which takes about 5 seconds to process. If the camera service sends multiple images to the worker, I want to only process the last one.

Should I even use temporal for this? Does it make sense to cancel a workflow and start a new one each time?

2

The standard approach is not to send a signal to the workflow about an update. If the workflow is busy executing some activity, it will queue the signal in its variable. When an activity is done, the workflow executes the next activity based on the queued signals according to its business logic. For example, it can decide to process only the last received signal.

It is common to send signals to a such workflow using the SignalWithStart API. This function first starts a workflow if not yet running.

Note that this pattern doesn’t work for use cases when a single workflow needs to process a high rate (more than a couple per second) of updates over a long period.