Temporalio/admin-tools using docker to create cassandra schema

Hi,

I am trying to use temporalio/admin-tools to create schema in cassandra, can you please let me know how I can do it, what is the docker command I need to use?

Thanks,
Kasi

1 Like

Hi Kasi, thank you for the question!

This section of Temporal Helm Chart README describes using temporal-cassandra-tool for initializing Temporal keyspaces in Cassandra.

And here is a fragment of how our CI/CD pipelines do this, via the admin-tools container running in the instance’s kubernetes cluster.

     # Setup keyspaces.
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 temporal-cassandra-tool create-Keyspace -k temporal"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 CASSANDRA_KEYSPACE=temporal temporal-cassandra-tool setup-schema -v 0.0"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 CASSANDRA_KEYSPACE=temporal ./temporal-cassandra-tool update -schema-dir /etc/temporal/schema/cassandra/temporal/versioned"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 temporal-cassandra-tool create-Keyspace -k temporal_visibility"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 CASSANDRA_KEYSPACE=temporal_visibility ./temporal-cassandra-tool setup-schema  -v 0.0"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 CASSANDRA_KEYSPACE=temporal_visibility ./temporal-cassandra-tool update -schema-dir /etc/temporal/schema/cassandra/visibility/versioned"

I hope this helps!

Thank you,
Mark.

Thanks mark, ideally I want to run the temporal-cassandra-tool from the admin-tools docker image using docker run --rm command, can you please help?

Ah! I don’t think we have any automation that does this specific operation, but I think you should be able to craft your docker run command line to do this, something along the lines of

$ docker run -rm -it -e CASSANDRA_HOST="cassandra-h1" -e < ... > --entrypoint /bin/bash  temporalio/admin-tools -c temporal-cassandra-tool create-Keyspace -k temporal
1 Like

Thanks a ton, will try it out

2 Likes

There is a setup schema setup-schema -v 0.0 and update schema with schema directory update-schema -d $SCHEMA_DIR how do we achieve this?

Thanks,

Kasi

I might be misreading the question, but if you are looking for the schema files inside of admin-tool docker image, see the snippet I included earlier, which includes complete paths (e. g. /etc/temporal/schema/cassandra/temporal/versioned).

Example:

     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 CASSANDRA_KEYSPACE=temporal ./temporal-cassandra-tool update -schema-dir /etc/temporal/schema/cassandra/temporal/versioned"

Thanks Mark, no you did not misread the question, using the docker the way we create the keySpace, we should also be able to create and update schema, the only issue is the update schema which has a schema directory which is dependent on the TEMPORAL_HOME, in the case of Docker how do we specify the temporal home

Oh interesting, I didn’t realize our schemas have a dependency on $TEMPORAL_HOME env var. How are you running schema setup tools? Maybe I could repro this.

And in any case, I think admin-tools already has this env var set to /etc/temporal:

$ docker run -it --entrypoint /bin/bash  temporalio/admin-tools -c env | grep TEMPORAL_HOME
TEMPORAL_HOME=/etc/temporal 

Great!!! will try it out

1 Like

Mark, temporal-cassandra-tool is complaining as if no arguments were passed to it

Hmmm… I wonder if i can reproduce this problem on my machine. Can you paste the command line that you are using, and the error are you seeing?

PS Also check out the snippet from earlier on this thread – notice how it is supplying environment variables. If those vars are not there, temporal-cassandra-tool will fallback on default values when possible, or fail. Make sure your code supplies those params.

     # Setup keyspaces.
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 temporal-cassandra-tool create-Keyspace -k temporal"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 CASSANDRA_KEYSPACE=temporal temporal-cassandra-tool setup-schema -v 0.0"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 CASSANDRA_KEYSPACE=temporal ./temporal-cassandra-tool update -schema-dir /etc/temporal/schema/cassandra/temporal/versioned"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 temporal-cassandra-tool create-Keyspace -k temporal_visibility"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 CASSANDRA_KEYSPACE=temporal_visibility ./temporal-cassandra-tool setup-schema  -v 0.0"
     kubectl exec -it services/temporaltest-admintools -- bash -c "CASSANDRA_HOST=cassandra.default.svc.cluster.local CASSANDRA_PORT=9042 CASSANDRA_KEYSPACE=temporal_visibility ./temporal-cassandra-tool update -schema-dir /etc/temporal/schema/cassandra/visibility/versioned"

Thanks Mark, I did, also what does the -c option do, cannot find a suitable explanation, I gave all the env params

ah, -c is just a param to bash, indicating that the string that follows is the command that needs to run in bash in the container.

https://linux.die.net/man/1/bash

Not sure what is happening there, but if you give me repro steps, I will try to repro and debug this.

Hey Mark,

Thanks a lot, here is the command.

docker run --rm -it -e CASSANDRA_HOST=“xxxxxxx” -e CASSANDRA_PORT=9042 -e CASSANDRA_USER=user1 -e CASSANDRA_PASSWORD=‘xxxxxx#2020’ --entrypoint /bin/sh dockerproxy.aexp.com/temporalio/admin-tools:0.26.0 -c temporal-cassandra-tool

the output is running the temporal-cassandra-tool with --help command.

Thanks,

Kasi

Thank you Kasi.

I am able to run your command line (I just changed the docker image name, to point to public dockerhub), and it prints the help screen:

$ docker run --rm -it -e CASSANDRA_HOST=“xxxxxxx” -e CASSANDRA_PORT=9042 -e CASSANDRA_USER=user1 -e CASSANDRA_PASSWORD=‘xxxxxx#2020’ --entrypoint /bin/sh temporalio/admin-tools:0.26.0 -c temporal-cassandra-tool
NAME:
   temporal-cassandra-tool - Command line tool for temporal cassandra operations

USAGE:
   temporal-cassandra-tool [global options] command [command options] [arguments...]

This is what I would expect – running the tool without params should generate the help screen.

If I start passing parameters from the keyspace creation sequence (Temporalio/admin-tools using docker to create cassandra schema), the command attempts to do the thing, and fails:

$ docker run --rm -it -e CASSANDRA_HOST=“xxxxxxx” -e CASSANDRA_PORT=9042 -e CASSANDRA_USER=user1 -e CASSANDRA_PASSWORD=‘xxxxxx#2020’ --entrypoint /bin/sh temporalio/admin-tools:0.26.0 -c "temporal-cassandra-tool create-Keyspace -k temporal"
2020/08/05 15:52:13 gocql: dns error: lookup “xxxxxxx”: no such host
2020/08/05 15:52:13 error creating Keyspace:gocql: unable to create session: failed to resolve any of the provided hostnames

This is also what I expect, since I am providing made up values for CASSANDRA_HOST, and I don’t have a cassandra host running at xxxxxxx.

I am assuming that if I pass in real values for CASSANDRA_* env variables, the command would work as expected.

Is this not what you are seeing?

Thanks a lot, Mark,I missed the quotes around the command

1 Like