Anyway - have figured it out
Current cluster should have a remote cluster block describing which certificate to present while communicating to the remote cluster
remoteClusters:
<remote cluster FQDN>:
server:
requireClientAuth: true
certFile: /etc/temporal/remote-certificates/server.pem
keyFile: /etc/temporal/remote-certificates/server.key
clientCaFiles:
- /etc/temporal/remote-certificates/ca.pem
client:
serverName: <remote cluster FQDN>
rootCaFiles:
- /etc/temporal/remote-certificates/ca.pem
Remove cluster should have either a catch-all TLS server configuration which require client to present a common mTLS cert or the host override like this to auth on per client basis for the specific call
frontend:
...
hostOverrides:
<remote cluster FQDN>:
requireClientAuth: true
certFile: /etc/temporal/remote-certificates/server.pem
keyFile: /etc/temporal/remote-certificates/server.key
clientCaFiles:
- /etc/temporal/remote-certificates/ca.pem