Ordering Guarantee of messages / Sequencing

Community Support
1

We have use cases for ordering of messages.
Needed to understand if there is any case of ordering/sequencing of messages with Temporal?
Similar to Kafka topic partition, is there anything on temporal for sequencing.

As per documentation by temporal: it is not provided, can somehow confirm this or is there some workaround for this

→ do not have any ordering guarantees
→ Task Queues support server-side throttling, which enables you to limit the Task dispatching rate to the pool of Worker Processes while still supporting Task dispatching at higher rates when spikes happen.

3

@PARIKSHIT it is hard to give recommendation without understanding your requirements.

  1. Do you need global ordering? Or per some business level ID like a customer?
  2. What is the maximum rate of events per such ID/partition?
  3. Why do you need the ordering in the first place?
4

Hi Maxim,

We need to processes messages in our queue in a defined order: Since all messages update the same document. If all the messages are processed at once, we land up into optimistic locking exception on the same document. Hence we were using a Kafka topic partition to guarantee ordering of messages.
But now we want to move to Temporal.

Rate of events: x > 0 and x < 2000 (These events/updates can be triggered concurrently, we need to control the processing on the consuming side)

I am trying a POC with temporal currently, where i have defined multiple activiy methods: @ActivityMethod which gets called under a @WorkflowMethod when a worker picks up a message.
Here i see that the acitivities are always executed in sequence. Even I bring down the worker and then bring it up and replay the workflow again, it starts from the activity it left off.
Just wondering is this the way to potentially order the execution of events, Or do we have better way in Temporal to order the events?

5

Is 2000 messages per second for a single document or across all the documents?

6

Hi @maxim

Yes, in a worst case scenario, we can have upto 100 messages coming to the backend at once.

7

To the same document?

8

Yes this can happen to the same document

9

The usual pattern is to have a workflow per business entity (document in your case) and send signals to it. Then, the workflow can process them in order. But this design doesn’t work for high update rates. If you need 100 requests/second to the same workflow, then it wouldn’t work. If you can rate limit them to something like 20 then it could work.

10

Understood.

Additionally If I were to create multiple workflows for all the updates (1 workflow per update which would be carrying the JSON payload to update single document), can I have ordering at a workflow level

11

No, we don’t support ordering across workflows yet.