Difference Between Archival and Inline Archival

Community Support
,
1

In the service/worker/archiver/client.go,
I see that there is an archival method.
In addition to that, there is also archiveVisibilityInline, archiveHistoryInline.

When I tried to backtrace the inline methods, it took me to the recordWorkflowClosed method.

Questions

  1. What’s the difference between Archival and Inline Archival and when exactly are the two called.
  2. I found that, as soon as the workflow is closed, the visibility data is archived the next moment. Even though the retention period is 1 day. Want to understand, when exactly does the archival really happen? Is it right after the workflow or a few hours before the retention period comes to an end?
2

  1. For history archival, Inline flag is set if workflow history size is less than TimerProcessorHistoryArchivalSizeLimit (default 5mb, can set in dynamicconfig). @Vitaly may have more info on this.

  2. History archival is done after the retention period. Visibility archival (when enabled) happens right after a running workflow closes. This is a current limitation that imo should be addressed in future archival updates. Archived data should be checked/accessed after the retention period only in order to have both history and visibility data available.

3

Thanks, Tihomir,
on the same topic, what is the purpose of sendArchivalSignal?

4

My understanding is that it’s the signal sent to archival workflow that includes the ArchiveRequest information so it can start archiving a particular wf. @Vitaly if you have more info on this please add :slight_smile:

5

What are the complications if I increase this to 10mb?

6

Checking with the server team to get their recommendations on this question. Tried locally and seems it did not cause any issues, but best for you to test as well. Please report any issues that you might find.

7

tihomir,

does this inline archival start after the workflow has been marked as completed ?

8

It starts after the retention period for a completed workflow.

Increasing the inline archival limit might put memory pressure on the history service and cause transfer queue processing delays. So you are on your own when changing it. Please test rigorously before deploying any such changes to production.