Unable to port-forward to temporal pod deployed in the k8s cluster

in the k8s cluster i’ve a running pod which is temporal service (1 instance) everything configured and connected properly (postgres, web-ui ) even runned a hello world example just to confirm everything is correct and all worked.
But im unable to port-forward directly to pod and run with tctl the commands.

> kubectl port-forward temporal-7469d47869-wxq6z 7233:7233
Forwarding from 127.0.0.1:7233 -> 7233
Forwarding from [::1]:7233 -> 7233
Handling connection for 7233
E0706 10:08:02.297540   72508 portforward.go:400] an error occurred forwarding 7233 -> 7233: error forwarding port 7233 to pod ....., uid : exit status 1: 2021/07/06 08:08:02 socat[27669] E connect(5, AF=2 127.0.0.1:7233, 16): Connection refused
tctl cluster health
Error: Unable to get "temporal.api.workflowservice.v1.WorkflowService" health check status.
Error Details: rpc error: code = Unavailable desc = connection closed
('export TEMPORAL_CLI_SHOW_STACKS=1' to see stack traces)

Questions:
What do i need to configure to allow it to accept connections?
Where should i bind the server ?

Thank you for your time.

Hi Gntem,

My apologies for the late response. My name is Jordan and I will be taking over this incident.

Can you double-check your kubectl version you were using in for this case? Also, can you check whether your pod has restarted while port-forward is running? If that happens, the behavior from kubectl 1.23.0 and later is for the kubectl port-forward command to log an error saying “lost connection to pod” and exit.

Here is a reference to a similar error that I found: Port-forward drops connection to pod after first connection · Issue #1169 · kubernetes/kubectl · GitHub

I look forward to your response. Please let me know if you run into any errors or mishaps.

Best Regards,
Jordan