Cadence connecting with AWS Keyspace

I am trying out AWS keyspace service for cadence cassandra store. When trying to using the cadence-cassandra-tool for schema migration i see two issues

  1. Partitioner Issue:
    Error: update the token ring due to error: Unsupported partitioner ‘com.amazonaws.cassandra.DefaultPartitioner’
  2. Consistency level:
    Error: consistency level ANY is not supported for this operation. Supported consistency levels are: ONE, LOCAL_QUORUM, LOCAL_ONE

Steps to reproduce the behavior:

  1. docker run --rm -it ubercadence/server:0.11.0 bash
  2. export the following env variable for cadence-cassandra-tool:
    export CASSANDRA_HOST=
    export CASSANDRA_PORT=
    export CASSANDRA_USER=
    export CASSANDRA_PASSWORD=
    export CASSANDRA_ENABLE_TLS=
    export CASSANDRA_TLS_VERIFY_HOST=
    export CASSANDRA_TLS_CA=
    export CADENCE_KEYSPACE=cadence
    export VISIBILITY_KEYSPACE=cadence_visibility
  3. cadence-Cassandra-tool setup-schema -v 0.0

Additional context:

I could not find any pointers for Partitioner issue. AWS documents mentions that they support gocql but only basic example provided, which does not talk about setting the partitioner. Based on the research i have done, i think gcql will only accepts following https://github.com/gocql/gocql/blob/cd92c541a0b5013df76fd0f980d978b76839f9a9/token.go#L156
and not sure how that gets abstracted for AWS Keyspace.

For consistency level issue, AWS Keyspace does not support Any and only support ONE, LOCAL_QUORUM, LOCAL_ONE as show in the error. When i checked in cadence-cassandra-tool i see that defaultconsistency as ALL as seen here

Any help would be greatly appreciated.

Thank you.

It looks like Keyspaces don’t support ALL as the schema is updated asynchronously. You can try to change the consistency level in the cassandra-tool to something else and see if it is going to work.

I found this discussion related to the partitioner.

1 Like

Hi Maxim,

Thanks for your quick response. I am following the same thread on amazon and it doesn’t have any info that i can try out. I will check if AWS support can help on this.

Regarding the changing the consistency level in the cassandra-tool , i get following error when trying to build locally with make bins. Can you please let me know if there is any additional required for building the tools.

git submodule update --init --recursive
thriftrw --plugin=yarpc --pkg-prefix=github.com/uber/cadence/.gen/go/ --out=.gen/go idls/thrift/cadence.thrift
Failed to generate code: failed to generate code for “C:\go\src\cadence\idls\thrift\cadence.thrift”: failed to generate code for “idl embedding”: could not parse generated code: cadence.go:9:28: illegal character U+0072 ‘r’ in escape sequence (and 4 more errors):
package thriftrw

            // ThriftModule represents the IDL file used to generate this package.
            var ThriftModule = &thriftreflect.ThriftModule {
                    Name: "cadence",
                    Package: "github.com\uber\cadence\.gen\go\cadence",
                    FilePath: "cadence.thrift",
                    SHA1: "e6751c109262ef7584dbec4c934a7d8e2dbe8123",
                    Includes: []*thriftreflect.ThriftModule {
                                            shared.ThriftModule,
                                    },
                    Raw: rawIDL,
            }

Thank you for your help.

It looks like a Windows specific issue. Do you have access to some Unix based machine?

Also would you try to build Temporal to see if it has a similar problem?

tools build for cadence and temporal is good on Mac(GOOS=darwin and GOARCH=amd64) but fail on windows and linux_amd64 . Attached the images in next comment due to post limitation

Thanks! Could you file an issue with details? It will help us to prioritize fixing this.