Temporal UI does not show archived workflows

Community Support
, , , ,
1

We enabled the archival for the cluster by setting this

server:
  enabled: true
  image:
    repository: temporalio/server
    tag: 1.21.5
  replicaCount: 1
  frontend:
    replicaCount: 1
  history:
    replicaCount: 1
  matching:
    replicaCount: 1
  worker:
    replicaCount: 1
  config:
    numHistoryShards: 16
    persistence:
      default:
        driver: "sql"
        sql:
          driver: "mysql8"
          host: "mysqllocal"
          port: 3306
          database: temporal
          user: root
          password: TemporalLocalMysqlTest123*
          maxConns: 20
          maxConnLifetime: "1h"
      visibility:
        driver: "sql"
        sql:
          driver: "mysql8"
          host: "mysqllocal"
          port: 3306
          database: temporal_visibility
          user: root
          password: TemporalLocalMysqlTest123*
          maxConns: 20
          maxConnLifetime: "1h"
  archival:
    history:
      state: "enabled"
      enableRead: true
      provider:
        filestore:
          fileMode: "0666"
          dirMode: "0766"
    visibility:
      state: "enabled"
      enableRead: true
      provider:
        filestore:
          fileMode: "0666"
          dirMode: "0766"
  namespaceDefaults:
    archival:
      history:
        state: "enabled"
        URI: "file:///tmp/temporal_archival/development"
      visibility:
        state: "enabled"
        URI: "file:///tmp/temporal_vis_archival/development"

Now in the namespace creation we have something like this

temporal operator namespace create --retention 1h --description 'Default namespace for sample service.' --history-archival-state enabled --visibility-archival-state enabled sample

we started few workflows and after the workflow completes we can see it in the workflow tab in the UI but not in the archive tab, and after couple of hours the workflow vanished from the workflow tab and it is not visible anywhere not even in the archive tab. Where are the workflows going ? why was it not archived ? we can see the archived files in the containers running something like this

kubectl exec -it pod/temporaldev-history-56c8d99959-8dvst bash -n temporaltest 
temporaldev-history-56c8d99959-8dvst:/etc/temporal$ cd /tmp/temporal_archival/development
temporaldev-history-56c8d99959-8dvst:/tmp/temporal_archival/development$ ls
13401993648341629848112907477214672046307440682199035548453_0.history   13401993648341629848154844513571057837306983859194921534913_0.history
13401993648341629848113118102869232220086472925130346424842_0.history   13401993648341629848177810928981828490966029814556947300621_0.history
13401993648341629848115582299840645356963922516828897260219_0.history   13401993648341629848180174586308258625192692670647861478596_0.history
13401993648341629848121355741058984871322428339163145085373_0.history   13401993648341629848941668685940622363810568900357782135342_0.history
134019936483416298481247265980575159657311668888450469171281_0.history  17344751046234800315243191562323785789715220876736843339497_0.history
134019936483416298481498330365938944825116591692805656484603_0.history  17344751046234800315649420603628501998613397172443626822597_0.history

The same thing if i deploy in docker locally, the archival works as soon as the workflow is completed. Strange !! as the docs specifically says this here : Self-hosted Archival setup | Temporal Documentation

Archival is not supported when running Temporal through Docker and is disabled by default when installing the system manually and when deploying through [helm charts](https://github.com/temporalio/helm-charts/blob/master/templates/server-configmap.yaml) (but can be enabled in the [config](https://github.com/temporalio/temporal/blob/master/config/development.yaml)).
2

Yep, we had the same problem. And I raise a pr to fix it. But it seems a lot of thing to do…

3

I ran this command from within the frontend container and i get this

tctl --ns sample wf show --wid SR-archival-1
Error: Failed to get history on workflow id: SR-archival-1, run id: .
Error Details: sql: no rows in result set

and then did this

temporal workflow list --archived=true --namespace sample

Any idea why it is not able to find the workflow even in the archival status ? looks like the workflow data is all gone …

4

Because according to you config –retention 1h , the closed workflow will only be stored in 1h. Then will it be move in to the archived store (below you set) and deleted from the default store.

    archival:
      history:
        state: "enabled"
        URI: "file:///tmp/temporal_archival/development"

For this I think maybe there is another thing to fix…

temporal workflow list --archived=true --namespace sample
5

Similar issues.

6

Does the archival work for local for you ? the local archival is not working for me, i mean i see the history files in the given location but its’ not querable using tctl nor can i see it in the archived tab in UI.