Unconfigured table closed_executions

Community Support
,
1

Trying to setup a cassandra backed temporal in k8s dev env, have services up and running, but getting unconfigured db errors in web: unconfigured table closed_executions or service":“frontend”,“error”:“ListClosedWorkflowExecutions operation failed. Error: unconfigured table closed_executions”

Per helm chart instructions, I have done the following and wonder what is missing for me. Thanks!
export CASSANDRA_HOST=cassandra_host
export CASSANDRA_PORT=9042
export CASSANDRA_USER=cassandra_user
export CASSANDRA_PASSWORD=cassandra_user_password

./temporal-cassandra-tool create-Keyspace -k temporal
CASSANDRA_KEYSPACE=temporal ./temporal-cassandra-tool setup-schema -v 0.0
CASSANDRA_KEYSPACE=temporal ./temporal-cassandra-tool update -schema-dir schema/cassandra/temporal/versioned

./temporal-cassandra-tool create-Keyspace -k temporal_visibility
CASSANDRA_KEYSPACE=temporal_visibility ./temporal-cassandra-tool setup-schema -v 0.0
CASSANDRA_KEYSPACE=temporal_visibility ./temporal-cassandra-tool update -schema-dir schema/cassandra/visibility/versioned

From create key space result seems those two tables are created.
2021/06/16 22:47:42 Creating Keyspace temporal_visibility using SimpleStrategy with RF=1
2021/06/16 22:47:47 Starting schema setup, config=&{SchemaFilePath: InitialVersion:0.0 Overwrite:false DisableVersioning:false}
2021/06/16 22:47:47 Setting up version tables
2021/06/16 22:47:51 Setting initial schema version to 0.0
2021/06/16 22:47:51 Updating schema update log
2021/06/16 22:47:51 Schema setup complete
2021/06/16 22:47:54 UpdateSchemeTask started, config=&{DBName: TargetVersion: SchemaDir:schema/cassandra/visibility/versioned IsDryRun:false}
2021/06/16 22:47:55 ---- Executing updates for version 1.0 ----
2021/06/16 22:47:55 CREATE TABLE open_executions (namespace_id uuid,namespace_partition int,workflow_id text,run_id uuid,start_time timestamp,execution_time timestamp,workflow_type_name text,memo blob,encoding text,task_queue text,PRIMARY KEY ((namespace_id, namespace_partition), start_time, run_id)) WITH CLUSTERING ORDER BY (start_time DESC)AND COMPACTION = {‘class’: ‘org.apache.cassandra.db.compaction.LeveledCompactionStrategy’,‘tombstone_threshold’: 0.4}AND GC_GRACE_SECONDS = 60;
2021/06/16 22:47:56 CREATE INDEX open_by_workflow_id ON open_executions (workflow_id);
2021/06/16 22:47:58 CREATE INDEX open_by_type ON open_executions (workflow_type_name);

2021/06/16 22:48:00 CREATE TABLE closed_executions (namespace_id uuid,namespace_partition int,workflow_id text,run_id uuid,start_time timestamp,execution_time timestamp,close_time timestamp,status int, workflow_type_name text,history_length bigint,memo blob,encoding text,task_queue text,PRIMARY KEY ((namespace_id, namespace_partition), close_time, run_id)) WITH CLUSTERING ORDER BY (close_time DESC)AND COMPACTION = {‘class’: ‘org.apache.cassandra.db.compaction.LeveledCompactionStrategy’}AND GC_GRACE_SECONDS = 172800;
2021/06/16 22:48:01 CREATE INDEX closed_by_workflow_id ON closed_executions (workflow_id);
2021/06/16 22:48:03 CREATE INDEX closed_by_close_time ON closed_executions (close_time);
2021/06/16 22:48:05 CREATE INDEX closed_by_type ON closed_executions (workflow_type_name);
2021/06/16 22:48:07 CREATE INDEX closed_by_status ON closed_executions (status);
2021/06/16 22:48:09 ---- Done ----
2021/06/16 22:48:09 Schema updated from 0.0 to 1.0

2

can you try using cqlsh verifying temporal_visibility keyspace exists and closed_executions table exists?
seems like a cassandra issue, no a temporal issue

3

Thanks Wenquan, seems no table created in keyspace_name = ‘temporal-visibility’, although casandra tool say so. What is the right way to create schema for visibility?

Currently I am doing this, and it only work for temporal keyspace:
export CASSANDRA_HOST=temporal-cassandra-dev.service.consul
export CASSANDRA_PORT=9042
export CASSANDRA_USER=cassandra
export CASSANDRA_PASSWORD=xxx
./temporal-cassandra-tool create-Keyspace -k temporal_visibility
CASSANDRA_KEYSPACE=temporal_visibility ./temporal-cassandra-tool setup-schema -v 0.0
CASSANDRA_KEYSPACE=temporal_visibility ./temporal-cassandra-tool update -schema-dir schema/cassandra/visibility/versioned

2021/06/17 11:04:06 Creating Keyspace temporal_visibility using SimpleStrategy with RF=1
2021/06/17 11:04:07 Starting schema setup, config=&{SchemaFilePath: InitialVersion:0.0 Overwrite:false DisableVersioning:false}
2021/06/17 11:04:07 Setting up version tables
2021/06/17 11:04:08 Current database schema version 1.0 is greater than initial schema version 0.0. Skip version upgrade
2021/06/17 11:04:08 Schema setup complete
2021/06/17 11:04:09 UpdateSchemeTask started, config=&{DBName: TargetVersion: SchemaDir:schema/cassandra/visibility/versioned IsDryRun:false}
2021/06/17 11:04:09 found zero updates from current version 1.0
2021/06/17 11:04:09 UpdateSchemeTask done

4

the schema tools ^ seems to suggest there is nothing to be done for temporal_visibility keyspace since visibility is already at 1.0 version (no more schema changes)

but you also mentioned seems no table created in keyspace_name = ‘temporal-visibility’

can you check if you are using the same cassandra cluster?

5

I only have one cassandra cluster and since I have ran Creating Keyspace temporal_visibility cmd before, so second time I run it there is nothing to update. But in both cases, no table come back from SELECT * FROM system_schema.tables WHERE keyspace_name = ‘temporal-visibility’;

I have also see cassandra log when first time creating visibility tables.
INFO 19:00:01 Waiting for gossip to settle before accepting client requests… │
│ INFO 19:00:02 Initializing temporal_visibility.closed_executions │
│ INFO 19:00:02 Initializing temporal_visibility.closed_executions.closed_by_close_time │
│ INFO 19:00:02 Initializing temporal_visibility.closed_executions.closed_by_status │
│ INFO 19:00:02 Initializing temporal_visibility.closed_executions.closed_by_type │
│ INFO 19:00:02 Initializing temporal_visibility.closed_executions.closed_by_workflow_id │
│ INFO 19:00:02 Initializing temporal_visibility.open_executions │
│ INFO 19:00:02 Initializing temporal_visibility.open_executions.open_by_type │
│ INFO 19:00:02 Initializing temporal_visibility.open_executions.open_by_workflow_id │
│ INFO 19:00:03 Initializing temporal_visibility.schema_update_history │
│ INFO 19:00:03 Initializing temporal_visibility.schema_version │
│ INFO 19:00:09 No gossip backlog; proceeding │
│ INFO 19:00:09 Netty using native Epoll event loop │
│ INFO 19:00:09 Using Netty Version: [netty-buffer=netty-buffer-4.0.39.Final.38bdf86, netty-codec=netty-codec-4.0.39.Final.38bdf86, netty-codec-haproxy=netty-codec-haproxy-4.0.39.Final.38bdf86, │
│ INFO 19:00:09 Starting listening for CQL clients on /0.0.0.0:9042 (unencrypted)… │
│ INFO 19:00:09 Not starting RPC server as requested. Use JMX (StorageService->startRPCServer()) or nodetool (enablethrift) to start it │
│ WARN 19:00:11 Trigger directory doesn’t exist, please create it and try again. │
│ INFO 19:00:11 Created default superuser role ‘cassandra’ │
│ INFO 19:00:28 Handshaking version with /10.32.46.22

6

temporal_visibility vs temporal-visibility

7

my bad, turns out service is looking for temporal-visibility vs cassandra tool create temporal_visibility, changing that fix it, thanks!