Regular Context Deadline Exceeded error when executing a workflow


I’ve been encountering the context deadline exceeded error regularly while using the ExecuteWorkflow method. I call it with a custom context.WithTimeout where the timeout is set to 1 minute. However, the error occurs 10 seconds after making the call in all cases.

Additionally, the temporal-history logs the context deadline exceeded error along with the following stack trace:*zapLogger).Error

And the temporal-worker’s stack trace is the following:*zapLogger).Error

No errors were found in the temporal-frontend part during the time when the error occurred.

The execution of a workflow roughly happens this way:

opts := client.StartWorkflowOptions{
	ID:                  uuid.NewString(),
	WorkflowTaskTimeout: time.Second * 30,
	TaskQueue:           "tx_retry_task_queue",

ctxWithTimeout, cancel := context.WithTimeout(ctx, time.Minute)
defer cancel()

_, err := rs.temporal.ExecuteWorkflow(ctxWithTimeout, opts, temporal.RetryTransactionWorkflowV2Name, tx)
if err != nil {
	// error handling

I’m running all of this within a Kubernetes environment. Could this issue be related to networking problems? May it be possible that the 10-second timeout is derived from the gRPC maximum timeout setting? Would it be possible to workaround the issue by using a RetryPolicy with some retries when executing a workflow?

Temporal Server Version: v1.19.0
Go Temporal SDK Version: v1.21.1

Will a workflow execution be retried if a context deadline exceeded error occurs, provided that I specify a RetryPolicy? Or do I need a custom retry logic for that?

UPD: custom retrying didn’t help

I am experiencing a very similar issue to this, would love to hear thoughts on how I could debug this further.

hi did any one of you find a solution to this. I have had the same issue with java sdk while running in a k8s environment. Found many posts related to load balancer idle time and keep alive time in sdk but still could not get rid of this error.