Ensuring last user input is processed last (FIFO method)

I’ve been reading the docs on the Task Queues, where it says “Task Queues do not have any ordering guarantees.”

Since I let users change their app settings through the REST API, and task queues don’t have ordering guarantees, I can’t see the reliability to always have the last user input handled last, to ensure I have their last settings saved in the Database, for example. Is this out of Temporal’s scope?

An example would be the user changing their e-mail address. On their form submit, I would send a “POST” request to the REST API.

The route handler would do two things—execute the Temporal workflow to do the permanent storage of the user’s input into Database, and in-memory (Redis) storage for quick responsiveness.

The route handler would not wait for the workflow results—it would only execute (start) the workflow to get the potential error in adding to the task queue. If Temporal does not return an error on execution, it would add the change to Redis and answer the client.

The thing is, if there are a lot of changes, and the Task queue gets loaded, I can’t ensure their last change is the one in the permanent Database storage if the Task queue has no ordering guarantees. This can lead to potential mismatch of permanent storage, in-memory storage, and the feedback the user gets in their UI (app).

Put down to super simple example, I could ask the user for their favorite number, and they could spam the API with [1, 2, 1, 2, 1, 2] (ignore the throttling for this).

The REST API handler would execute workflows with 1-> 2-> 1-> 2-> 1-> 2, which would add them all to the Task queue, and the workers would run them with activities to save the preference to the Database.

How can I ensure the last UPDATE in the Database is with the number 2? With no ordering guarantee it would potentially do a random order 1-> 1-> 2-> 2-> 2-> 1, leaving me with the preference of 1 instead of 2.

I also thought of adding the timestamp of the last change in the Database and then compare the values, to discard any earlier changes than the last, but that brings me back to the solution I could craft with the queue system, such as RabbitMQ and their ordered queues. Timestamp wherever I might have a change. :man_shrugging:

You can create a workflow per “resource” (for example per user in your scenario). This workflow should receive updates via signals, and handle them one by one (in the order received). This way you can control updates sequentially.

This approach is good if your rate of updates per user workflow is low. The aggregate rate can be high tho.

Also take a look at this signal reordering sample which I think is related to your requirements.

2 Likes