Could I know the significance of taskQueuePartitions in presence of high load on a single task Queue? From @maxim ’s talk on How to Design a workflow engine from first principles , I inferred that a single task queue must reside on a single shard (which is why local transfer queues exist…to transfer…

I think there is some terminology confusion. Temporal history service divides workflow id space into numHistoryShards shards. Each shard contains a transfer queue. This queue implements an outbox pattern as tasks are inserted into it using the same transaction that updates the workflow state. Task…

Scaling Task Queues | Correlation between numHistoryShards, taskQueuePartitions, number of matching service hosts

Community Support

maxim January 13, 2023, 7:45pm 6

You have to configure 100 partitions to support 10k tasks per second.

Estimating the right configuration values of the temporal services

Topic		Replies	Views
Communication between worker process and matching service Community Support	5	142	June 11, 2024
Temporal Shards ( configured in 'shards' table ) Community Support java-sdk	6	114	February 10, 2025
numHistoryShards cluster migration Community Support performance	12	1322	June 14, 2021
History Service CPU usage Community Support	5	922	March 26, 2021
Temporal framework performance evaluation Community Support performance	6	2115	August 20, 2022

Scaling Task Queues | Correlation between numHistoryShards, taskQueuePartitions, number of matching service hosts

Related topics