Hello! I’m trying to connect to use RDS as temporal
database, but it crashes on
sql schema version compatibility check failed: unable to read DB schema version keyspace/database: temporal error: no usable database connection found
error. Temporal correctly connects to DB at first, checks migrations and eventually executes them. DB is created, migrations are performed, tables exist in DB. But then this error occurs and I have no idea why as it’s able to connect to execute the migrations.
I’m using Docker compose which works correctly with “local” PG, but not with RDS. this my Docker compose file:
temporal:
container_name: temporal
restart: unless-stopped
environment:
- DB=postgres12
- DB_PORT=5432
- POSTGRES_SEEDS=***.eu-west-1.rds.amazonaws.com
- POSTGRES_USER=postgres
- POSTGRES_PWD=***
- POSTGRES_TLS_ENABLED=true
- POSTGRES_TLS_DISABLE_HOST_VERIFICATION=true
- DYNAMIC_CONFIG_FILE_PATH=config/dynamicconfig/development-sql.yaml
image: temporalio/auto-setup:1.27@sha256:501a883b1c4a4168eb5000c368495e7eb7106dbadefc75b7fd886be0108456b9
ports:
- 7233:7233
volumes:
- ./volumes/temporal-dynamicconfig:/etc/temporal/config/dynamicconfig
extra_hosts:
- "host.docker.internal:host-gateway"
labels:
kompose.volume.type: configMap
temporal-admin-tools:
container_name: temporal-admin-tools
restart: unless-stopped
depends_on:
- temporal
environment:
- TEMPORAL_ADDRESS=temporal:7233
- TEMPORAL_CLI_ADDRESS=temporal:7233
- TEMPORAL_CLI_SHOW_STACKS=1
image: temporalio/admin-tools:1.25.2-tctl-1.18.1-cli-1.1.1@sha256:da0c7a7982b571857173ab8f058e7f139b3054800abb4dcb100445d29a563ee8
stdin_open: true
tty: true
entrypoint: /etc/temporal/entrypoint.sh
extra_hosts:
- "host.docker.internal:host-gateway"
healthcheck:
test: ["CMD", "tctl", "workflow", "list"]
interval: 1s
timeout: 5s
retries: 30
volumes:
- ./scripts/mirror-name-search.sh:/etc/temporal/entrypoint.sh
temporal-ui:
container_name: temporal-ui
restart: unless-stopped
depends_on:
- temporal
environment:
- TEMPORAL_ADDRESS=temporal:7233
- TEMPORAL_CORS_ORIGINS=http://localhost:3000
- TEMPORAL_CSRF_COOKIE_INSECURE=true
image: temporalio/ui:2.35.1@sha256:a98e49436dab613aaea77e72da4003f606045deaca066b706c421eeb2c1acad4
extra_hosts:
- "host.docker.internal:host-gateway"
ports:
- 8085:8080
The full logs when Docker compose starts:
temporal | TEMPORAL_ADDRESS is not set, setting it to 192.168.107.3:7233
temporal | PostgreSQL started.
temporal | Setup PostgreSQL schema.
temporal-ui | 2025/03/03 10:27:55 Loading config; env=docker,configDir=config
temporal-ui | 2025/03/03 10:27:55 Loading config files=[config/docker.yaml]
temporal-ui | 2025/03/03 10:27:55 Loading config; env=docker,configDir=config
temporal-ui | 2025/03/03 10:27:55 Loading config files=[config/docker.yaml]
temporal-ui |
temporal-ui | ____ __
temporal-ui | / __/___/ / ___
temporal-ui | / _// __/ _ \/ _ \
temporal-ui | /___/\__/_//_/\___/ v4.9.0
temporal-ui | High performance, minimalist Go web framework
temporal-ui | https://echo.labstack.com
temporal-ui | ____________________________________O/_______
temporal-ui | O\
temporal-ui | ⇨ http server started on [::]:8080
peerdb-server | 2025-03-03T10:27:56.481070Z INFO NOTICE: relation "refinery_schema_history" already exists, skipping
peerdb-server | 2025-03-03T10:27:56.777997Z INFO current version: 42
peerdb-server | 2025-03-03T10:27:56.778073Z INFO no migrations to apply
peerdb-server | 2025-03-03T10:27:56.778908Z INFO Listening on 0.0.0.0:9900
peerdb-server | 2025-03-03T10:27:56.778924Z INFO MIRROR commands enabled
peerdb-server | 2025-03-03T10:27:56.778930Z INFO connecting to flow server at grpc://flow_api:8112/grpc
temporal | 2025-03-03T10:27:57.065Z INFO Starting schema setup {"config": {"SchemaFilePath":"","SchemaName":"","InitialVersion":"0.0","Overwrite":false,"DisableVersioning":false}, "logging-call-at": "/home/runner/work/docker-builds/docker-builds/temporal/tools/common/schema/setuptask.go:64"}
temporal | 2025-03-03T10:27:57.066Z DEBUG Setting up version tables {"logging-call-at": "/home/runner/work/docker-builds/docker-builds/temporal/tools/common/schema/setuptask.go:74"}
temporal | 2025-03-03T10:27:57.262Z DEBUG Current database schema version 1.15 is greater than initial schema version 0.0. Skip version upgrade {"logging-call-at": "/home/runner/work/docker-builds/docker-builds/temporal/tools/common/schema/setuptask.go:135"}
temporal | 2025-03-03T10:27:57.262Z INFO Schema setup complete {"logging-call-at": "/home/runner/work/docker-builds/docker-builds/temporal/tools/common/schema/setuptask.go:150"}
temporal | 2025-03-03T10:27:57.784Z INFO UpdateSchemaTask started {"config": {"DBName":"","TargetVersion":"","SchemaDir":"/etc/temporal/schema/postgresql/v12/temporal/versioned","SchemaName":"","IsDryRun":false}, "logging-call-at": "/home/runner/work/docker-builds/docker-builds/temporal/tools/common/schema/updatetask.go:106"}
temporal | 2025-03-03T10:27:57.885Z DEBUG Schema Dirs: [] {"logging-call-at": "/home/runner/work/docker-builds/docker-builds/temporal/tools/common/schema/updatetask.go:214"}
temporal | 2025-03-03T10:27:57.885Z DEBUG found zero updates from current version 1.15 {"logging-call-at": "/home/runner/work/docker-builds/docker-builds/temporal/tools/common/schema/updatetask.go:136"}
temporal | 2025-03-03T10:27:57.886Z INFO UpdateSchemaTask done {"logging-call-at": "/home/runner/work/docker-builds/docker-builds/temporal/tools/common/schema/updatetask.go:129"}
temporal | 2025-03-03T10:27:58.971Z INFO Starting schema setup {"config": {"SchemaFilePath":"","SchemaName":"","InitialVersion":"0.0","Overwrite":false,"DisableVersioning":false}, "logging-call-at": "/home/runner/work/docker-builds/docker-builds/temporal/tools/common/schema/setuptask.go:64"}
temporal | 2025-03-03T10:27:58.972Z DEBUG Setting up version tables {"logging-call-at": "/home/runner/work/docker-builds/docker-builds/temporal/tools/common/schema/setuptask.go:74"}
temporal | 2025-03-03T10:27:59.170Z DEBUG Current database schema version 1.9 is greater than initial schema version 0.0. Skip version upgrade {"logging-call-at": "/home/runner/work/docker-builds/docker-builds/temporal/tools/common/schema/setuptask.go:135"}
temporal | 2025-03-03T10:27:59.170Z INFO Schema setup complete {"logging-call-at": "/home/runner/work/docker-builds/docker-builds/temporal/tools/common/schema/setuptask.go:150"}
temporal | 2025-03-03T10:27:59.675Z INFO UpdateSchemaTask started {"config": {"DBName":"","TargetVersion":"","SchemaDir":"/etc/temporal/schema/postgresql/v12/visibility/versioned","SchemaName":"","IsDryRun":false}, "logging-call-at": "/home/runner/work/docker-builds/docker-builds/temporal/tools/common/schema/updatetask.go:106"}
temporal | 2025-03-03T10:27:59.774Z DEBUG Schema Dirs: [] {"logging-call-at": "/home/runner/work/docker-builds/docker-builds/temporal/tools/common/schema/updatetask.go:214"}
temporal | 2025-03-03T10:27:59.774Z DEBUG found zero updates from current version 1.9 {"logging-call-at": "/home/runner/work/docker-builds/docker-builds/temporal/tools/common/schema/updatetask.go:136"}
temporal | 2025-03-03T10:27:59.774Z INFO UpdateSchemaTask done {"logging-call-at": "/home/runner/work/docker-builds/docker-builds/temporal/tools/common/schema/updatetask.go:129"}
temporal | Temporal CLI address: 192.168.107.3:7233.
temporal | 2025/03/03 10:27:59 Loading config; env=docker,zone=,configDir=config
temporal | 2025/03/03 10:27:59 Loading config files=[config/docker.yaml]
temporal | {"level":"info","ts":"2025-03-03T10:27:59.836Z","msg":"Build info.","git-time":"2025-02-21T21:43:57.000Z","git-revision":"675b2d5a8a2c9c6051f618e3728c6082fa925078","git-modified":false,"go-arch":"arm64","go-os":"linux","go-version":"go1.23.2","cgo-enabled":false,"server-version":"1.27.0","debug-mode":false,"logging-call-at":"/home/runner/work/docker-builds/docker-builds/temporal/cmd/server/main.go:175"}
temporal | {"level":"info","ts":"2025-03-03T10:27:59.837Z","msg":"dynamic config changed for the key: limit.maxidlength oldValue: nil newValue: { constraints: {} value: 255 }","logging-call-at":"/home/runner/work/docker-builds/docker-builds/temporal/common/dynamicconfig/file_based_client.go:361"}
temporal | {"level":"info","ts":"2025-03-03T10:27:59.837Z","msg":"dynamic config changed for the key: system.forcesearchattributescacherefreshonread oldValue: nil newValue: { constraints: {} value: true }","logging-call-at":"/home/runner/work/docker-builds/docker-builds/temporal/common/dynamicconfig/file_based_client.go:361"}
temporal | {"level":"info","ts":"2025-03-03T10:27:59.837Z","msg":"dynamic config changed for the key: frontend.enableupdateworkflowexecution oldValue: nil newValue: { constraints: {} value: true }","logging-call-at":"/home/runner/work/docker-builds/docker-builds/temporal/common/dynamicconfig/file_based_client.go:361"}
temporal | {"level":"info","ts":"2025-03-03T10:27:59.837Z","msg":"Updated dynamic config","logging-call-at":"/home/runner/work/docker-builds/docker-builds/temporal/common/dynamicconfig/file_based_client.go:207"}
temporal | {"level":"warn","ts":"2025-03-03T10:27:59.837Z","msg":"Not using any authorizer and flag `--allow-no-auth` not detected. Future versions will require using the flag `--allow-no-auth` if you do not want to set an authorizer.","logging-call-at":"/home/runner/work/docker-builds/docker-builds/temporal/cmd/server/main.go:205"}
temporal | time=2025-03-03T10:27:59.838 level=ERROR msg="failed reaching server: last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 192.168.107.3:7233: connect: connection refused\""
temporal | Waiting for Temporal server to start...
temporal | [Fx] PROVIDE fx.Lifecycle <= go.uber.org/fx.New.func1()
temporal | [Fx] PROVIDE fx.Shutdowner <= go.uber.org/fx.(*App).shutdowner-fm()
temporal | [Fx] PROVIDE fx.DotGraph <= go.uber.org/fx.(*App).dotGraph-fm()
temporal | [Fx] PROVIDE *temporal.ServerImpl <= go.temporal.io/server/temporal.NewServerFxImpl()
temporal | [Fx] PROVIDE *temporal.serverOptions <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE chan interface {} <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE temporal.synchronizationModeParams <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE *config.Config <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE *config.PProf <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE log.Config <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE resource.ServiceNames <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE resource.NamespaceLogger <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE resolver.ServiceResolver <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE client.AbstractDataStoreFactory <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE visibility.VisibilityStoreFactory <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE searchattribute.Mapper <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE []grpc.UnaryServerInterceptor <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE authorization.Authorizer <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE authorization.ClaimMapper <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE authorization.JWTAudienceMapper <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE map[primitives.ServiceName]static.Hosts <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE log.Logger <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE client.FactoryProvider <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE dynamicconfig.Client <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE encryption.TLSConfigProvider <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE *client.Config <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE client.Client <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE metrics.Handler <= go.temporal.io/server/temporal.ServerOptionsProvider()
temporal | [Fx] PROVIDE archiver.ArchivalMetadata <= go.temporal.io/server/common/resource.ArchivalMetadataProvider()
temporal | [Fx] PROVIDE tasks.TaskCategoryRegistry <= go.temporal.io/server/temporal.TaskCategoryRegistryProvider()
temporal | [Fx] PROVIDE client.FactoryProviderFn <= go.temporal.io/server/temporal.PersistenceFactoryProvider()
temporal | [Fx] PROVIDE *temporal.ServicesMetadata[group = "services"] <= go.temporal.io/server/temporal.HistoryServiceProvider()
temporal | [Fx] PROVIDE *temporal.ServicesMetadata[group = "services"] <= go.temporal.io/server/temporal.MatchingServiceProvider()
temporal | [Fx] PROVIDE *temporal.ServicesMetadata[group = "services"] <= go.temporal.io/server/temporal.FrontendServiceProvider()
temporal | [Fx] PROVIDE *temporal.ServicesMetadata[group = "services"] <= go.temporal.io/server/temporal.InternalFrontendServiceProvider()
temporal | [Fx] PROVIDE *temporal.ServicesMetadata[group = "services"] <= go.temporal.io/server/temporal.WorkerServiceProvider()
temporal | [Fx] PROVIDE *cluster.Config <= go.temporal.io/server/temporal.ApplyClusterMetadataConfigProvider()
temporal | [Fx] PROVIDE config.Persistence <= go.temporal.io/server/temporal.ApplyClusterMetadataConfigProvider()
temporal | [Fx] PROVIDE *dynamicconfig.Collection <= go.temporal.io/server/common/dynamicconfig.init.func1()
temporal | [Fx] PROVIDE pingable.Pingable[group = "deadlockDetectorRoots"] <= fx.Annotate(go.temporal.io/server/common/dynamicconfig.init.func2(), fx.ResultTags(["group:\"deadlockDetectorRoots\""])
temporal | [Fx] PROVIDE *pprof.PProfInitializerImpl <= go.temporal.io/server/common/pprof.NewInitializer()
temporal | [Fx] PROVIDE []trace.SpanExporter <= go.temporal.io/server/temporal.init.func2()
temporal | [Fx] SUPPLY []temporal.ServerOption
temporal | [Fx] RUN supply: stub([]temporal.ServerOption) in 4.75µs
temporal | [Fx] RUN provide: go.temporal.io/server/temporal.ServerOptionsProvider() in 101.082576ms
temporal | [Fx] Error returned: received non-nil error from function "go.temporal.io/server/temporal".ServerOptionsProvider
temporal | /home/runner/work/docker-builds/docker-builds/temporal/temporal/fx.go:182:
temporal | sql schema version compatibility check failed: unable to read DB schema version keyspace/database: temporal error: no usable database connection found
temporal | [Fx] ERROR Failed to initialize custom logger: could not build arguments for function "go.uber.org/fx".(*module).constructCustomLogger.func2
temporal | /home/runner/go/pkg/mod/go.uber.org/fx@v1.23.0/module.go:294:
temporal | failed to build fxevent.Logger:
temporal | could not build arguments for function "go.temporal.io/server/temporal".init.func7
temporal | /home/runner/work/docker-builds/docker-builds/temporal/temporal/fx.go:1008:
temporal | failed to build log.Logger:
temporal | received non-nil error from function "go.temporal.io/server/temporal".ServerOptionsProvider
temporal | /home/runner/work/docker-builds/docker-builds/temporal/temporal/fx.go:182:
temporal | sql schema version compatibility check failed: unable to read DB schema version keyspace/database: temporal error: no usable database connection found
temporal | Unable to create server. Error: could not build arguments for function "go.uber.org/fx".(*module).constructCustomLogger.func2 (/home/runner/go/pkg/mod/go.uber.org/fx@v1.23.0/module.go:294): failed to build fxevent.Logger: could not build arguments for function "go.temporal.io/server/temporal".init.func7 (/home/runner/work/docker-builds/docker-builds/temporal/temporal/fx.go:1008): failed to build log.Logger: received non-nil error from function "go.temporal.io/server/temporal".ServerOptionsProvider (/home/runner/work/docker-builds/docker-builds/temporal/temporal/fx.go:182): sql schema version compatibility check failed: unable to read DB schema version keyspace/database: temporal error: no usable database connection found.
temporal exited with code 1
temporal | PostgreSQL started.
temporal | Setup PostgreSQL schema.
temporal-admin-tools | time=2025-03-03T10:28:01.036 level=ERROR msg="failed reaching server: last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 192.168.107.3:7233: connect: connection refused\""
temporal-admin-tools | time=2025-03-03T10:28:01.050 level=ERROR msg="failed reaching server: last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 192.168.107.3:7233: connect: connection refused\""
Thank you for any help! I’m really stuck there. Thank you!