ActivityWorker: context deadline exceeded

Community Support
1

Hey,
I have an activity wherein I use googleclient to Download a file, and return the file as byte.
I keep getting “context deadline exceeded” errors in it.

Here is the activity code:

func (a *Activities) DownloadFile(ctx context.Context, fileId string, fileName string) ([]byte, error) {
	logger := activity.GetLogger(ctx)
	logger.Info("Downloading file from Google Drive...")
	timeValue, _ := ctx.Deadline()
	logger.Info(timeValue.GoString())
	data, _ := a.FileSrv.Files.Get(fileId).Download()
	defer data.Body.Close()

	body, err := ioutil.ReadAll(data.Body)

	if err != nil {
		logger.Error("Error while reading response body")
		return nil, err
	}

	return body, nil
}

Here is the workflow function::

func FileProcessingWorkflow(ctx workflow.Context, record *CsvRecord) (string, error) {
	logger := workflow.GetLogger(ctx)
	options := workflow.ActivityOptions{
		StartToCloseTimeout: time.Second * 60,
		RetryPolicy: &temporal.RetryPolicy{
			MaximumAttempts: 3,
		},
	}

	var a *Activities
	logger.Info("Starting activity execution...")
	ctx = workflow.WithActivityOptions(ctx, options)

	var result []byte
	//TODO: Fetch user id and bank statement drive ID here
	logger.Info("Executing download statement Activity")
	err := workflow.ExecuteActivity(ctx, a.DownloadFile, record.FileId, record.FileName).Get(ctx, &result)
	if err != nil {
		return "", err
	}
    //process the []byte further
}

Any pointers where I might be going wrong
I am running the basic docker-compose version on an M1 Mac

2

image
image2368×186 70.1 KB

3

The error is not related to the activity execution. It is a workflow task timeout. What is the size of the file? Temporal doesn’t support very large payloads as activity results.

4

I’m trying to download a PDF from Google Drive, and then pass it on as a byte array to the next activity.
PDF size ranges from 2MB to 8MB…
6.6 MB in this particular case

5

The maximum payload size is 2 MB. So it is definitely not going to work. The recommended workarounds:

  • Cache result locally on the worker and use a session to route the next activity to the same process.
  • Pass a pointer to the blob store to the next activity
  • Paginate by returning partial results. Note that the total history size cannot exceed 50 MB.
1 Like
6

Oh Ok, didn’t know that.

  1. Anywhere in the docs I can read about size limits?
  2. Probably naive question, where should I catch these type of errors in the Workflow code ?
7
  1. Temporal Server self-hosted production deployment | Temporal Documentation
  2. I think workflow should get an error that contains the relevant information. You should also see ActivityTaskFailed event in the workflow execution history.