Could I know the significance of taskQueuePartitions in presence of high load on a single task Queue? From @maxim ’s talk on How to Design a workflow engine from first principles , I inferred that a single task queue must reside on a single shard (which is why local transfer queues exist…to transfer…

I think there is some terminology confusion. Temporal history service divides workflow id space into numHistoryShards shards. Each shard contains a transfer queue. This queue implements an outbox pattern as tasks are inserted into it using the same transaction that updates the workflow state. Task…

Scaling Task Queues | Correlation between numHistoryShards, taskQueuePartitions, number of matching service hosts

Community Support

maxim January 13, 2023, 7:45pm 6

You have to configure 100 partitions to support 10k tasks per second.

Estimating the right configuration values of the temporal services

Topic		Replies	Views
numHistoryShards cluster migration Community Support performance	12	1268	June 14, 2021
History Service CPU usage Community Support	5	888	March 26, 2021
Temporal framework performance evaluation Community Support performance	6	2085	August 20, 2022
numHistoryShards modify Community Support	6	1173	April 6, 2023
Temporal studying - various questions Community Support	5	1354	February 9, 2021

Scaling Task Queues | Correlation between numHistoryShards, taskQueuePartitions, number of matching service hosts

Related topics