Temporal Worker unable to talk to Internal Frontend, receiving Request Unauthorized

[Bharadwaj_Embar]

Temporal Server Version: v1.20.2

Hello,

We are running Temporal with TLS (not mTLS) and Authorization (default Authorizer and default ClaimMapper). After upgrading to v1.18.6 we hit this issue, so we turned off the authorizer and claim mapper. We upgraded to v1.20.2 and enabled them again (both values set to default).

We receive the follow errors from the worker:

In the worker code in scanner.go, we see that we instantiate a GetSystemClient() and there are no HeadersProviders passed in the SDK client options. Because of this, we don’t pass an authorization token and ultimately it ends up failing when calling the internal-frontend with Request unauthorized error.

Ref: temporal/scanner.go at v1.20.2 · temporalio/temporal · GitHub

While instantiating the internal-frontend, we set Claim Mapper to use the NoopClaimMapper implementation by default, but the authorizer is the one set in configuration (which in our case is default)

Ref: temporal/fx.go at v1.20.2 · temporalio/temporal · GitHub

As we understand, we have atleast two options if not more:

1. Set the internal-frontend to use NoopAuthorizer by default
2. Pass the HeadersProvider when creating the worker client

For 1, other teams might have an authorizer that they would like to use. But considering that all internal worker requests goes through the internal-frontend, this may be a good option.

Please do let us know of your thoughts or if there is a missing configuration/misconfiguration from our side. This will definitely affect users who have enabled authorization.

[tihomir]

I think you might be looking at using this pr which should be available in next release.

[tianruizeng]

Does temporal worker talk with frontend using TLS connection?