Temporal worker failing to connect v1.22

Community Support
1

hi
we deploy Temporal on k8s , we enabled mtls and it seems that all worked , when we add authorization the temporal worker start to fail with error .
“level”:“error”,“ts”:“2023-11-27T16:51:45.083Z”,“msg”:“error starting temporal-sys-tq-scanner-workflow workflow”,“service”:“worker”,“error”:“Request unauthorized.”,“logging-call-at”:“scanner.go:289”,“stacktrace”:“go.temporal.io/server/common/log.(*zapLogger).Error\n\t/home/builder/temporal/common/log/zap_logger.go:156\ngo.temporal.io/server/service/worker/scanner.(*Scanner).startWorkflow\n\t/home/builder/temporal/service/worker/scanner/scanner.go:289\ngo.temporal.io/server/service/worker/scanner.(*Scanner).startWorkflowWithRetry.func1\n\t/home/builder/temporal/service/worker/scanner/scanner.go:259\ngo.temporal.io/server/common/backoff.ThrottleRetryContext\n\t/home/builder/temporal/common/backoff/retry.go:145\ngo.temporal.io/server/service/worker/scanner.(*Scanner).startWorkflowWithRetry\n\t/home/builder/temporal/service/worker/scanner/scanner.go:258”}

{“level”:“error”,“ts”:“2023-11-27T16:51:45.083Z”,“msg”:“error starting temporal-sys-history-scanner-workflow workflow”,“service”:“worker”,“error”:“Request unauthorized.”,“logging-call-at”:“scanner.go:289”,“stacktrace”:“go.temporal.io/server/common/log.(*zapLogger).Error\n\t/home/builder/temporal/common/log/zap_logger.go:156\ngo.temporal.io/server/service/worker/scanner.(*Scanner).startWorkflow\n\t/home/builder/temporal/service/worker/scanner/scanner.go:289\ngo.temporal.io/server/service/worker/scanner.(*Scanner).startWorkflowWithRetry.func1\n\t/home/builder/temporal/service/worker/scanner/scanner.go:259\ngo.temporal.io/server/common/backoff.ThrottleRetryContext\n\t/home/builder/temporal/common/backoff/retry.go:145\ngo.temporal.io/server/service/worker/scanner.(*Scanner).startWorkflowWithRetry\n\t/home/builder/temporal/service/worker/scanner/scanner.go:258”}

all other services are working also history

2

Can you share your generated static config? Think this section would be important to check: https://github.com/temporalio/temporal/blob/main/docker/config_template.yaml#L264-L274

3

see our config below , except the temporal worker that fail on " Request unauthorized" all is working .

global:
tls:
internode:
server:
certFile: /mnt/secrets-store/cluster.cert
keyFile: /mnt/secrets-store/cluster.key
requireClientAuth: true
clientCaFiles:
- /mnt/secrets-store/ca.cert
client:
serverName: temporal-auth-worker-headless
disableHostVerification: false
rootCaFiles:
- /mnt/secrets-store/ca.cert
frontend:
server:
certFile: /mnt/secrets-store/cluster.cert
keyFile: /mnt/secrets-store/cluster.key
requireClientAuth: true
clientCaFiles:
- /mnt/secrets-store/ca.cert
client:
serverName: temporal-auth-worker-headless
disableHostVerification: false
rootCaFiles:
- /mnt/secrets-store/ca.cert

authorization:
authorizer: default
claimMapper: default

    permissionsClaimName: roles
    jwtKeyProvider:
      keySourceURIs:
        - https://login.microsoftonline.com/......
4

Hi friend,

I also encountered the same issue after enabling the authorization, did you get it addressed?

1 Like
5

Hi andyng,
I managed to fix this issue by enabling the internalFrontend in my helm values.yaml file like so. This allows the worker pods to use the internal front end which isn’t so picky about authorization tokens.

server:
  internalFrontend:
    enabled: true