Support JWT only from the Temporal UI and not the SDK

From what I can gather from testing the only way to restrict access to the Temporal UI is to enable ClaimMapper and Authorizer on the Temporal server.

This causes the all frontend requests to be challenge by inspecting a token for claims. In this case a client application using the SDK.

Is it possible to enforce authorisation on UI but only expect mutual authorisation using the SDK?

Thanks in advance