Install and migrate temporal without helm chart

Community Support
1

Hi,

we are planning to use temporal in our production environment but we have some costraints and so we probably need some suggestions about the correct approach.
We cannot use kubernetes at the moment and so we are not able to use the provided helm chart.

Saying that, our idea is:

INSTALLATION:
1 - install a separate postgres db (it will be use also for other purposes).
2 - Install 3 nodes with docker-compose instances of temporal containing:
- temporalio/auto-setup:${version} (this is configured to be connected to the existing DB)
- temporalio/web:${version}
The container “temporalio/admin-tools:${version}” should not be required in the installed nodes because it should be used only for db migration and cli “tctl”. Could you confirm this?

MIGRATION:
this should be the approach:
1 - Migrate DB: run a custom script (I shared it at the end of post) by using the desired “temporalio/admin-tools” version (TEMPORAL_ADMIN_TOOLS_VERSION). This should be compatible with older server version
2 - Rolling Update of docker-compose (server and web). This should be at 0 downtime

I’m trying to understand the limitations of this approach; for example I read that the “Archival is not supported when running Temporal via docker-compose”.

  • Could you suggest a workaround for this?
  • There are other limitations to the docker-compose approach?
  • Have you got general guides or best practices or other solutions?

Great Thanks for your time and support
Marco

#################################################################################
#!/bin/bash
export TEMPORAL_ADMIN_TOOLS_VERSION=${TEMPORAL_ADMIN_TOOLS_VERSION:-1.6.2}
SCHEMA_DIR=/etc/temporal/schema/postgresql/v96/temporal/versioned
VISIBILITY_SCHEMA_DIR=/etc/temporal/schema/postgresql/v96/visibility/versioned

POSTGRES_PWD="${POSTGRES_PWD:-temporal}"
{ export SQL_PASSWORD=$POSTGRES_PWD; } 2> /dev/null
export DBNAME="${DBNAME:-temporal}"
export VISIBILITY_DBNAME="${VISIBILITY_DBNAME:-temporal_visibility}"
export SQL_USER=${POSTGRES_USER:-temporal}
export SQL_PORT=${DB_PORT:-5432}
export SQL_HOST=${POSTGRES_SEEDS:-postgresql}
export SQL_PLUGIN=postgres

DOCKER_PARAMS="–env SQL_PLUGIN --env SQL_HOST --env SQL_PORT --env SQL_USER --env SQL_PASSWORD --rm --network temporal-network -it --entrypoint /bin/bash temporalio/admin-tools:$TEMPORAL_ADMIN_TOOLS_VERSION"

UPDATE

echo “>>>>>>>>>>>>>>>>>>>>>>> Update …”
docker run $DOCKER_PARAMS -c “temporal-sql-tool --db $DBNAME update-schema -d $SCHEMA_DIR”
docker run $DOCKER_PARAMS -c “temporal-sql-tool --db $VISIBILITY_DBNAME update-schema -d $VISIBILITY_SCHEMA_DIR”

#################################################################################

2

1 → looks good
2 → use temporalio/server, not temporalio/auto-setup for you production env, and manually setup the db schema: ref: https://github.com/temporalio/temporal/blob/v1.7.0/Makefile#L335

3

archival is currently an experimental feature.

4

Hi,
I tried to follow your suggestion by replacing the docker-compose image auto-setup with temporalio/server:1.7.0 but It seems that the server is not able to start due to:
“Unable to start server: sql schema version compatibility check failed: dial tcp 10.252.64.2:5432: connect: connection refused.”

There is probably a missing configuration parameter. Have you got a description of the required environment variables used to connect to postgres db?

BTW: regarding the archival feature, what do you mean that it is experimental? is it possible to use it in my case or I need to wait an offical release (in this case, have you got a plan?)

This is my docker-compose section

##########################à
temporal:
container_name: temporal
depends_on:
- postgresql
environment:
- DB=postgresql
- DB_PORT=5432
- POSTGRES_USER=temporal
- POSTGRES_PWD=temporal
- POSTGRES_SEEDS=postgresql
- DYNAMIC_CONFIG_FILE_PATH=config/dynamicconfig/development.yaml
image: temporalio/server:1.7.0
networks:
- temporal-network
ports:
- 7233:7233
volumes:
- ./dynamicconfig:/etc/temporal/config/dynamicconfig
##########################à

Great Thanks
Marco

5

could you try making sure 10.252.64.2 is the IP address of your PG setup, and can you verify that DB is ready by using psql?

6

Hi,
the DB host seems to be correct but the issue I see is the strange behaviour:
I firstly used the docker-compose file you provide https://raw.githubusercontent.com/temporalio/docker-compose/v1.7.0/docker-compose-postgres.yml (it contains image “temporalio/auto-setup”) and all works fine.
If I replace “temporalio/auto-setup” with “temporalio/server” (I tried both with and without environment variable “AUTO_SETUP=true”) it fails with the postgres connection issue.

So I suppose that there are some differences between the two images that make the different behaviour.

Great Thanks
Marco

8

Hi,
I understood the issue.
I found a post in community related to the fact that by setting the environment variable AUTO_SETUP=true to the temporalio/server its behaviour should be the same of auto-setup image.

I found in your Dockerfile that auto-setup calls start.sh with autosetup parameter

FROM temporal-server AS temporal-auto-setup
CMD /start.sh autosetup

The autosetup calls the schema setup

auto_setup() {
    wait_for_db
    if [ "$SKIP_SCHEMA_SETUP" != true ]; then
        setup_schema
    fi

    if [ "$SKIP_DEFAULT_NAMESPACE_CREATION" != true ]; then
        register_default_namespace &
    fi
}

For this reason by using temporal/server in docker-compose, the postgres DB is not setup and it fails.

Do you know why the env variable AUTO_SETUP was deprecated?

Great Thanks
Marco

9
  1. I do not recommend using auto setup (automatic setting up DB schema or upgrading DB) for prod use case. DB schema upgrade is relative rare, and should be monitored.
  2. AUTO_SETUP env is not part of our public interface, in fact, the env is removed before 1.0: Remove AUTO_SETUP env var by markmark206 · Pull Request #752 · temporalio/temporal · GitHub