Temporalio/admin-tools using docker to create cassandra schema

kasi · August 3, 2020, 6:40pm

Hi,

I am trying to use temporalio/admin-tools to create schema in cassandra, can you please let me know how I can do it, what is the docker command I need to use?

Thanks,
Kasi

markmark · August 3, 2020, 7:12pm

Hi Kasi, thank you for the question!

This section of Temporal Helm Chart README describes using temporal-cassandra-tool for initializing Temporal keyspaces in Cassandra.

And here is a fragment of how our CI/CD pipelines do this, via the admin-tools container running in the instance’s kubernetes cluster.

     # Setup keyspaces.
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 temporal-cassandra-tool create-Keyspace -k temporal"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 CASSANDRA_KEYSPACE=temporal temporal-cassandra-tool setup-schema -v 0.0"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 CASSANDRA_KEYSPACE=temporal ./temporal-cassandra-tool update -schema-dir /etc/temporal/schema/cassandra/temporal/versioned"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 temporal-cassandra-tool create-Keyspace -k temporal_visibility"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 CASSANDRA_KEYSPACE=temporal_visibility ./temporal-cassandra-tool setup-schema  -v 0.0"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 CASSANDRA_KEYSPACE=temporal_visibility ./temporal-cassandra-tool update -schema-dir /etc/temporal/schema/cassandra/visibility/versioned"

I hope this helps!

Thank you,
Mark.

kasi · August 3, 2020, 7:23pm

Thanks mark, ideally I want to run the temporal-cassandra-tool from the admin-tools docker image using docker run --rm command, can you please help?

markmark · August 3, 2020, 7:35pm

Ah! I don’t think we have any automation that does this specific operation, but I think you should be able to craft your docker run command line to do this, something along the lines of

$ docker run -rm -it -e CASSANDRA_HOST="cassandra-h1" -e < ... > --entrypoint /bin/bash  temporalio/admin-tools -c temporal-cassandra-tool create-Keyspace -k temporal

kasi · August 3, 2020, 7:47pm

Thanks a ton, will try it out

kasi · August 3, 2020, 9:22pm

There is a setup schema setup-schema -v 0.0 and update schema with schema directory update-schema -d $SCHEMA_DIR how do we achieve this?

Thanks,

Kasi

markmark · August 3, 2020, 9:41pm

I might be misreading the question, but if you are looking for the schema files inside of admin-tool docker image, see the snippet I included earlier, which includes complete paths (e. g. /etc/temporal/schema/cassandra/temporal/versioned).

Example:

     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 CASSANDRA_KEYSPACE=temporal ./temporal-cassandra-tool update -schema-dir /etc/temporal/schema/cassandra/temporal/versioned"

kasi · August 3, 2020, 9:48pm

Thanks Mark, no you did not misread the question, using the docker the way we create the keySpace, we should also be able to create and update schema, the only issue is the update schema which has a schema directory which is dependent on the TEMPORAL_HOME, in the case of Docker how do we specify the temporal home

markmark · August 3, 2020, 10:00pm

Oh interesting, I didn’t realize our schemas have a dependency on $TEMPORAL_HOME env var. How are you running schema setup tools? Maybe I could repro this.

And in any case, I think admin-tools already has this env var set to /etc/temporal:

$ docker run -it --entrypoint /bin/bash  temporalio/admin-tools -c env | grep TEMPORAL_HOME
TEMPORAL_HOME=/etc/temporal

kasi · August 3, 2020, 10:16pm

Great!!! will try it out

kasi · August 4, 2020, 5:50pm

Mark, temporal-cassandra-tool is complaining as if no arguments were passed to it

markmark · August 4, 2020, 9:43pm

Hmmm… I wonder if i can reproduce this problem on my machine. Can you paste the command line that you are using, and the error are you seeing?

markmark · August 4, 2020, 9:54pm

PS Also check out the snippet from earlier on this thread – notice how it is supplying environment variables. If those vars are not there, temporal-cassandra-tool will fallback on default values when possible, or fail. Make sure your code supplies those params.

     # Setup keyspaces.
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 temporal-cassandra-tool create-Keyspace -k temporal"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 CASSANDRA_KEYSPACE=temporal temporal-cassandra-tool setup-schema -v 0.0"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 CASSANDRA_KEYSPACE=temporal ./temporal-cassandra-tool update -schema-dir /etc/temporal/schema/cassandra/temporal/versioned"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 temporal-cassandra-tool create-Keyspace -k temporal_visibility"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 CASSANDRA_KEYSPACE=temporal_visibility ./temporal-cassandra-tool setup-schema  -v 0.0"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 CASSANDRA_KEYSPACE=temporal_visibility ./temporal-cassandra-tool update -schema-dir /etc/temporal/schema/cassandra/visibility/versioned"

kasi · August 4, 2020, 9:56pm

Thanks Mark, I did, also what does the -c option do, cannot find a suitable explanation, I gave all the env params

markmark · August 4, 2020, 10:00pm

ah, -c is just a param to bash, indicating that the string that follows is the command that needs to run in bash in the container.

https://linux.die.net/man/1/bash

Not sure what is happening there, but if you give me repro steps, I will try to repro and debug this.

kasi · August 5, 2020, 3:25pm

Hey Mark,

Thanks a lot, here is the command.

docker run --rm -it -e CASSANDRA_HOST=“xxxxxxx” -e CASSANDRA_PORT=9042 -e CASSANDRA_USER=user1 -e CASSANDRA_PASSWORD=‘xxxxxx#2020’ --entrypoint /bin/sh dockerproxy.aexp.com/temporalio/admin-tools:0.26.0 -c temporal-cassandra-tool

the output is running the temporal-cassandra-tool with --help command.

Thanks,

Kasi

markmark · August 5, 2020, 4:06pm

Thank you Kasi.

I am able to run your command line (I just changed the docker image name, to point to public dockerhub), and it prints the help screen:

$ docker run --rm -it -e CASSANDRA_HOST=“xxxxxxx” -e CASSANDRA_PORT=9042 -e CASSANDRA_USER=user1 -e CASSANDRA_PASSWORD=‘xxxxxx#2020’ --entrypoint /bin/sh temporalio/admin-tools:0.26.0 -c temporal-cassandra-tool
NAME:
   temporal-cassandra-tool - Command line tool for temporal cassandra operations

USAGE:
   temporal-cassandra-tool [global options] command [command options] [arguments...]

This is what I would expect – running the tool without params should generate the help screen.

If I start passing parameters from the keyspace creation sequence (Temporalio/admin-tools using docker to create cassandra schema), the command attempts to do the thing, and fails:

$ docker run --rm -it -e CASSANDRA_HOST=“xxxxxxx” -e CASSANDRA_PORT=9042 -e CASSANDRA_USER=user1 -e CASSANDRA_PASSWORD=‘xxxxxx#2020’ --entrypoint /bin/sh temporalio/admin-tools:0.26.0 -c "temporal-cassandra-tool create-Keyspace -k temporal"
2020/08/05 15:52:13 gocql: dns error: lookup “xxxxxxx”: no such host
2020/08/05 15:52:13 error creating Keyspace:gocql: unable to create session: failed to resolve any of the provided hostnames

This is also what I expect, since I am providing made up values for CASSANDRA_HOST, and I don’t have a cassandra host running at xxxxxxx.

I am assuming that if I pass in real values for CASSANDRA_* env variables, the command would work as expected.

Is this not what you are seeing?

kasi · August 5, 2020, 6:44pm

Thanks a lot, Mark,I missed the quotes around the command

Topic		Replies	Views
Temporal Helm Cassandra problem Community Support helm , cassandra	3	1248	April 6, 2021
How to automate the setup when use separated mysql (or any db)? Community Support mysql	14	2258	April 27, 2021
Errors while setting up Temporal on local environment using helm chart Community Support helm , cassandra	4	1427	October 8, 2021
How build live env not use docker Community Support mysql	6	779	November 24, 2021
Temporal server down - Cassandra version compatibility check failed no connections were made when creating the session Server Deployment cassandra	2	1095	November 14, 2023

Temporalio/admin-tools using docker to create cassandra schema

Related topics