Archival to s3 in helm charts

Hi Team,

What would be your suggestion for archival to s3 in helm charts.
Please share, If there is a doc for the feature else kindly share the idea to implement .

1 Like

Hey. Archival is supported in Helm but disabled by default. You can update the helm configuration to enable archival and we do support S3. We do have some various docs which could help:

If you still have more questions let me know.

Hi @ryland, Thanks for your reply.


in this URL, the config mentioned is for docker compose. In official helm values.yaml, I couldnt find the related configs. If I were to add a custom values. will the structure of the config yaml remain the same.

one more thing what do we mean by endpoint?
archival:
history:
state: “enabled”
enableRead: true
provider:
s3store:
region: …
endpoint: …

and URI would be s3:/// ? in
archival:
history:
state: “enabled”
URI: …

Hi @ryland,

(in the context of .29 helm chart)

I tried adding the value as mentioned in the below topic by @samar in the server-config.yaml

archival:
  history:
    state: "enabled"
    enableRead: true
    provider:
      s3store:
        region: "eu-central-1"
        endpoint: "https://<bucket_name>.s3.eu-central-1.amazonaws.com"   **>>>>> (1) is the format correct ??**
  visibility:
    state: "enabled"
    enableRead: true
    provider:
      s3store:
        region: "eu-central-1"
        endpoint: "https://<bucket_name>.s3.eu-central-1.amazonaws.com"  

namespaceDefaults:
  archival:
    history:
      state: "enabled"
      URI: "s3://<bucket_name>/history"   **>>>> (2) is the format correct ??**
    visibility:
      state: "enabled"
      URI: "s3://<bucket_name>/visibility"

the archival didnt work as expected, which led me to think of possibilities.

1: is the s3store endpoint format correct ??
2: is the URI format correct ??
3: if these two values are correct, where will the archival process get the authorisation to push data into the s3 ?? (I have attached s3-full-access policy in a role and attached it to the EKS cluster)

I checked the tctl namespace list cmd in the admintool pod which is clearly showing that the archival is still disabled even after passing the enabled value in the configmap.

image

Any lead on this regard would be helpful.

thanks and regards
Sandeep

1 Like

Hi Sandeep , did you check this link on setting up archival

for s3, may be this section will be more relevant,

1 Like