I’m running a multi-cluster setup across 2 (GCP) regions. Both have a pool of workers pointing at their local temporal cluster. I was surprised to find that workers connecting to the passive cluster are picking up tasks. Per another thread, I was under the impression this would not happen (our desired behavior).
I wanted to check if my understanding is correct - should workers connected to the passive cluster be receiving tasks? If so, any ideas what might be causing this to happen?
Here’s what our multi-cluster config looks like:
dcRedirectionPolicy: policy: "all-apis-forwarding" clusterMetadata: enableGlobalNamespace: true failoverVersionIncrement: 10 masterClusterName: "usc1" clusterInformation: usc1: enabled: true initialFailoverVersion: 1 rpcAddress: "temporal-frontend.usc1.staging.tock:7233" usw1: enabled: true initialFailoverVersion: 2 rpcAddress: "temporal-frontend.usw1.staging.tock:7233"
tctl output, both clusters seem to agree that usc1 is the active cluster (we are only using the default namespace):
$ tctl namespace describe creating config dir: /.config/temporalio creating config dir: /.config/temporalio Name: default Id: a235aa5a-7052-4792-93cc-b45b7d9d195c Description: OwnerEmail: NamespaceData: map[string]string(nil) State: Registered Retention: 720h0m0s ActiveClusterName: usc1 Clusters: usc1, usw1 HistoryArchivalState: Enabled IsGlobalNamespace: true