Kubernetes - Schema Upgrade Stuck with MySQL

Does anyone perhaps know why temporal with a first time installation is stuck on the schema upgrades?

It does not give any visible warnings or errors. Just times out and fails to complete.

MySQL Version: 8.0.35
Kubernetes: GKE v1.27.7

history.go:56: [debug] getting history for release temporal-beta
Release "temporal-beta" does not exist. Installing it now.
install.go:214: [debug] Original chart version: ""
install.go:231: [debug] CHART PATH: /home/xxxxx/tmpcode/temporal/temporal

client.go:142: [debug] creating 1 resource(s)
client.go:712: [debug] Watching for changes to Job temporal-beta-schema-setup with timeout of 5m0s
client.go:740: [debug] Add/Modify event for temporal-beta-schema-setup: ADDED
client.go:779: [debug] temporal-beta-schema-setup: Jobs active: 0, jobs failed: 0, jobs succeeded: 0
client.go:740: [debug] Add/Modify event for temporal-beta-schema-setup: MODIFIED
client.go:779: [debug] temporal-beta-schema-setup: Jobs active: 1, jobs failed: 0, jobs succeeded: 0
client.go:740: [debug] Add/Modify event for temporal-beta-schema-setup: MODIFIED
client.go:779: [debug] temporal-beta-schema-setup: Jobs active: 0, jobs failed: 0, jobs succeeded: 0
client.go:740: [debug] Add/Modify event for temporal-beta-schema-setup: MODIFIED
client.go:142: [debug] creating 1 resource(s)
client.go:712: [debug] Watching for changes to Job temporal-beta-schema-update with timeout of 5m0s
client.go:740: [debug] Add/Modify event for temporal-beta-schema-update: ADDED
client.go:779: [debug] temporal-beta-schema-update: Jobs active: 1, jobs failed: 0, jobs succeeded: 0
client.go:740: [debug] Add/Modify event for temporal-beta-schema-update: MODIFIED
client.go:779: [debug] temporal-beta-schema-update: Jobs active: 1, jobs failed: 0, jobs succeeded: 0
client.go:486: [debug] Starting delete for "temporal-beta-schema-update" Job
Error: failed pre-install: 1 error occurred:
        * timed out waiting for the condition


helm.go:84: [debug] failed pre-install: 1 error occurred:
        * timed out waiting for the condition

I used this post as my base: Temporal service one single database - #3 by wandrewchan

the schema pod output:

default-schema 2024-05-10T11:33:02.132Z INFO Starting schema setup {"config": {"SchemaFilePath":"","InitialVersion":"0.0","Overwrite":false,"Disa ││ default-schema 2024-05-10T11:33:02.134Z DEBUG Setting up version tables {"logging-call-at": "setuptask.go:68"} ││ default-schema 2024-05-10T11:33:02.149Z DEBUG Setting initial schema version to 0.0 {"logging-call-at": "setuptask.go:119"} ││ default-schema 2024-05-10T11:33:02.162Z DEBUG Updating schema update log {"logging-call-at": "setuptask.go:124"} ││ default-schema 2024-05-10T11:33:02.199Z INFO Schema setup complete {"logging-call-at": "setuptask.go:132"} ││ visibility-schema 2024-05-10T11:33:02.378Z INFO Starting schema setup {"config": {"SchemaFilePath":"","InitialVersion":"0.0","Overwrite":false,"D ││ visibility-schema 2024-05-10T11:33:02.378Z DEBUG Setting up version tables {"logging-call-at": "setuptask.go:68"} ││ visibility-schema 2024-05-10T11:33:02.388Z DEBUG Setting initial schema version to 0.0 {"logging-call-at": "setuptask.go:119"} ││ visibility-schema 2024-05-10T11:33:02.397Z DEBUG Updating schema update log {"logging-call-at": "setuptask.go:124"} ││ visibility-schema 2024-05-10T11:33:02.407Z INFO Schema setup complete {"logging-call-at": "setuptask.go:132"} ││ Stream closed EOF for app-beta-eu-temporal/temporal-beta-schema-setup-zcq4d (create-default-store) ││ Stream closed EOF for app-beta-eu-temporal/temporal-beta-schema-setup-zcq4d (create-visibility-store) ││ Stream closed EOF for app-beta-eu-temporal/temporal-beta-schema-setup-zcq4d (visibility-schema) ││ Stream closed EOF for app-beta-eu-temporal/temporal-beta-schema-setup-zcq4d (default-schema)

The schema update pod output:

Defaulted container "default-schema" out of: default-schema, visibility-schema

1 Like

Spent an entire night trying to figure this out also.