Migrating Elasticsearch Data from ES9.x to ES8.x

Community Support Server Deployment
1

Hello. This is a likely unique issue.

Facts:

  • We have self-hosted Temporal with docker-compose
  • Our visibility database is ES
  • When I set up ES for the visibility database, I erroneously installed ES 9.x, which was not obviously unsupported in the docs.
  • I also erroneously allowed auto setup run with: ES_VERSION=v7
  • This broke date range queries because of syntax changes in ES9.x
  • We have spun up an ES8.x server for fix this issue.
  • Temporal has been running for 3 weeks with ES9.x and we do not want to lose the data.

Redacted docker-compose.yml:

services:
temporal:
container_name: temporal
environment:

  • DB=postgres12
  • DB_PORT=5432
  • POSTGRES_USER=redacted
  • POSTGRES_PWD=redacted
  • POSTGRES_SEEDS=redacted
  • DB_NAME=temporal
  • DYNAMIC_CONFIG_FILE_PATH=config/dynamicconfig/development-sql.yaml
  • ENABLE_ES=true
  • ES_SEEDS=redacted
  • ES_PORT=9200
  • ES_VERSION=v7
  • ES_USER=redacted
  • ES_PWD=redacted
  • ES_SCHEME=https
  • PROMETHEUS_ENDPOINT=0.0.0.0:8000
  • TEMPORAL_ADDRESS=temporal:7233
  • TEMPORAL_CLI_ADDRESS=temporal:7233
    image: temporalio/auto-setup:latest
    networks:
  • temporal-network
    ports:
  • 7233:7233
  • 8000:8000
    volumes:
  • ./dynamicconfig:/etc/temporal/config/dynamicconfig

temporal-admin-tools:
container_name: temporal-admin-tools
depends_on:

  • temporal
    environment:
  • TEMPORAL_ADDRESS=temporal:7233
  • TEMPORAL_CLI_ADDRESS=temporal:7233
    image: temporalio/admin-tools:latest
    networks:
  • temporal-network
    stdin_open: true
    tty: true

temporal-ui:
container_name: temporal-ui
depends_on:

  • temporal
    environment:
  • TEMPORAL_ADDRESS=temporal:7233
  • TEMPORAL_CORS_ORIGINS=redacted
    image: temporalio/ui:latest
    networks:
  • temporal-network
    ports:
  • 8080:8080

Question:
What is a safe procedure to:

  • migrate existing data from old ES9 server to support ES8 server?
  • restart Temporal via docker compose?