Activity parameters and returns values impact on performance

Community Support
1

With reference to performance co-relation with activity parameters and return values mentioned below.

We are planning to use the iterator workflow pattern as mentioned by maxim in the thread. However, this pattern leads to an increase in the size of results and values being passed between activities.

So, I have the following queries on the same:

  1. Is there any recommendation on the upper cap of data to be passed between activities?
  2. How does temporal stores these values in the workflow execution history? Is there any multiplier impact here in co-relation to the number of attempts made for an activity?
2

Temporal does record activity inputs and outputs in the workflow history. Only the last result is stored in the history. Activity retries are not recorded in the history.

The hard cap on the activity size is 2MB. But it is for outliers as the whole history size cap is 50MB. We recommend keeping average activity inputs and outputs within a few dozen kilobytes.

The possible solutions for the large activity inputs and outputs:

  • Store results in a blob store like S3 and pass the reference as an activity result.
  • Store results in process and route next activity to the same process.
2 Likes
3

Thanks, this helps.

4

Hi @maxim,

I came across your recommendation to store the output of an activity in an S3 bucket. However, considering that the subsequent activity needs to download the data from the S3 bucket and may potentially upload another sizable file, it seems like this approach introduces some overhead with repetitive download and upload operations.

I’m curious if there’s a way to store this data in the run context instead. This way, subsequent activities could retrieve the required information directly from the context, potentially avoiding the need for repetitive downloading and uploading of files. I’d love to hear your thoughts on this approach and whether it aligns with best practices

5

I don’t know what “run context” is. Temporal doesn’t have such concept.

Have you considered:

  • Store results in process and route next activity to the same process.
6

HI @maxim thanks for your reply. I am not sure how I can store the results in the process and route next activity to the same process without without payload limit and informations privacy. I would like to have this information behind the scenes, that the next activity uses that, but without needing to pass this information in the payload.

7

Your activity stores the payload in the local in-process storage, for example, a hash map or filesystem. And passes a symbolic name of the stored object to the next activity. Please look at the fileprocessing sample that demonstrates this approach.

1 Like