Temporal worker failing to connect v1.22

Community Support
1

hi
we deploy Temporal on k8s , we enabled mtls and it seems that all worked , when we add authorization the temporal worker start to fail with error .
“level”:“error”,“ts”:“2023-11-27T16:51:45.083Z”,“msg”:“error starting temporal-sys-tq-scanner-workflow workflow”,“service”:“worker”,“error”:“Request unauthorized.”,“logging-call-at”:“scanner.go:289”,“stacktrace”:“go.temporal.io/server/common/log.(*zapLogger).Error\n\t/home/builder/temporal/common/log/zap_logger.go:156\ngo.temporal.io/server/service/worker/scanner.(*Scanner).startWorkflow\n\t/home/builder/temporal/service/worker/scanner/scanner.go:289\ngo.temporal.io/server/service/worker/scanner.(*Scanner).startWorkflowWithRetry.func1\n\t/home/builder/temporal/service/worker/scanner/scanner.go:259\ngo.temporal.io/server/common/backoff.ThrottleRetryContext\n\t/home/builder/temporal/common/backoff/retry.go:145\ngo.temporal.io/server/service/worker/scanner.(*Scanner).startWorkflowWithRetry\n\t/home/builder/temporal/service/worker/scanner/scanner.go:258”}

{“level”:“error”,“ts”:“2023-11-27T16:51:45.083Z”,“msg”:“error starting temporal-sys-history-scanner-workflow workflow”,“service”:“worker”,“error”:“Request unauthorized.”,“logging-call-at”:“scanner.go:289”,“stacktrace”:“go.temporal.io/server/common/log.(*zapLogger).Error\n\t/home/builder/temporal/common/log/zap_logger.go:156\ngo.temporal.io/server/service/worker/scanner.(*Scanner).startWorkflow\n\t/home/builder/temporal/service/worker/scanner/scanner.go:289\ngo.temporal.io/server/service/worker/scanner.(*Scanner).startWorkflowWithRetry.func1\n\t/home/builder/temporal/service/worker/scanner/scanner.go:259\ngo.temporal.io/server/common/backoff.ThrottleRetryContext\n\t/home/builder/temporal/common/backoff/retry.go:145\ngo.temporal.io/server/service/worker/scanner.(*Scanner).startWorkflowWithRetry\n\t/home/builder/temporal/service/worker/scanner/scanner.go:258”}

all other services are working also history

2

Can you share your generated static config? Think this section would be important to check: https://github.com/temporalio/temporal/blob/main/docker/config_template.yaml#L264-L274

3

see our config below , except the temporal worker that fail on " Request unauthorized" all is working .

global:
tls:
internode:
server:
certFile: /mnt/secrets-store/cluster.cert
keyFile: /mnt/secrets-store/cluster.key
requireClientAuth: true
clientCaFiles:
- /mnt/secrets-store/ca.cert
client:
serverName: temporal-auth-worker-headless
disableHostVerification: false
rootCaFiles:
- /mnt/secrets-store/ca.cert
frontend:
server:
certFile: /mnt/secrets-store/cluster.cert
keyFile: /mnt/secrets-store/cluster.key
requireClientAuth: true
clientCaFiles:
- /mnt/secrets-store/ca.cert
client:
serverName: temporal-auth-worker-headless
disableHostVerification: false
rootCaFiles:
- /mnt/secrets-store/ca.cert

authorization:
authorizer: default
claimMapper: default

    permissionsClaimName: roles
    jwtKeyProvider:
      keySourceURIs:
        - https://login.microsoftonline.com/......