Worker Failed to poll for task

Community Support
,
1

Hi all,

We have been seeing below warning logs in our worker application deployed on k8s. These logs appear at unexpected intervals, so couldn’t derive any patterns yet. Wanted to understand if anyone else has seen this error in the past and understand when do we get this error ?

Msg: Failed to poll for task
Error: unexpected HTTP status code received from server: 504 (Gateway Timeout)
Trace:go.temporal.io/sdk/internal.(*baseWorker).logPollTaskError
	/go/pkg/mod/go.temporal.io/sdk@v1.15.0/internal/internal_worker_base.go:353
go.temporal.io/sdk/internal.(*baseWorker).pollTask
	/go/pkg/mod/go.temporal.io/sdk@v1.15.0/internal/internal_worker_base.go:316
go.temporal.io/sdk/internal.(*baseWorker).runPoller
	/go/pkg/mod/go.temporal.io/sdk@v1.15.0/internal/internal_worker_base.go:279

~Nitesh

2

Think need more info, do you see errors from your frontend service pods? From any other temporal service pods?
Is your worker pod having issues? Do you get this during deployment/redeployment?
Does this warning transient or you see that it impacts your executions/performance?

3

Sure. Please find the answers inline.

Yes, it’s the worker pod apps polling temporal task queues.There is no issue while execution as such. Just the warning logs are unexpected. Hence, trying to understand the root cause,

No we get it post deployment as well.

Yes the warning is transient and there is no visible impact on performance/execution yet.

4

Temporal uses 60 second long poll to receive tasks. So make sure that the networking/load balancing you use doesn’t cut gRPC connections for these long running requests.

1 Like
5

Thanks, @maxim. I think this might be the issue. Let me update the setting at our end and revert back accordingly.