Archival to s3 in helm charts

Community Support
, ,
1

Hi Team,

What would be your suggestion for archival to s3 in helm charts.
Please share, If there is a doc for the feature else kindly share the idea to implement .

1 Like
2

Hey. Archival is supported in Helm but disabled by default. You can update the helm configuration to enable archival and we do support S3. We do have some various docs which could help:

If you still have more questions let me know.

3

Hi @ryland, Thanks for your reply.

in this URL, the config mentioned is for docker compose. In official helm values.yaml, I couldnt find the related configs. If I were to add a custom values. will the structure of the config yaml remain the same.

one more thing what do we mean by endpoint?
archival:
history:
state: “enabled”
enableRead: true
provider:
s3store:
region: …
endpoint: …

and URI would be s3:/// ? in
archival:
history:
state: “enabled”
URI: …

4

Hi @ryland,

(in the context of .29 helm chart)

I tried adding the value as mentioned in the below topic by @samar in the server-config.yaml

archival:
  history:
    state: "enabled"
    enableRead: true
    provider:
      s3store:
        region: "eu-central-1"
        endpoint: "https://<bucket_name>.s3.eu-central-1.amazonaws.com"   **>>>>> (1) is the format correct ??**
  visibility:
    state: "enabled"
    enableRead: true
    provider:
      s3store:
        region: "eu-central-1"
        endpoint: "https://<bucket_name>.s3.eu-central-1.amazonaws.com"  

namespaceDefaults:
  archival:
    history:
      state: "enabled"
      URI: "s3://<bucket_name>/history"   **>>>> (2) is the format correct ??**
    visibility:
      state: "enabled"
      URI: "s3://<bucket_name>/visibility"

the archival didnt work as expected, which led me to think of possibilities.

1: is the s3store endpoint format correct ??
2: is the URI format correct ??
3: if these two values are correct, where will the archival process get the authorisation to push data into the s3 ?? (I have attached s3-full-access policy in a role and attached it to the EKS cluster)

I checked the tctl namespace list cmd in the admintool pod which is clearly showing that the archival is still disabled even after passing the enabled value in the configmap.

image

Any lead on this regard would be helpful.

thanks and regards
Sandeep

1 Like
5

Hi Sandeep , did you check this link on setting up archival

for s3, may be this section will be more relevant,

1 Like
6

Hi @madhu @Sandeep_Paul

the docs to which you pointed here are not accessible anymore. Can you provide updated links in case the link answers the above questions ?

7

you could refer the latest docs How to set up Archival | Temporal Documentation

specific to s3 you can refer this sample from temporals github

1 Like
8

S3 archiver readme also has some relevant info for setup and search query limitations.
tctl namespace update command could also be used to enable/disable archival and set archival urls

1 Like
9

Thanks folks. It will be great if we can point the documentation towards the readme.