Not able to connect remote cassandra database, installing using helm chart

Hi Team,
I am trying to connect remote Cassandra DB for the temporal server while installing using the helm chart on GKE. the temporal server is running on one cluster and Cassandra db is up and running on another cluster. entering correct hostname and tls is also enabled (passing caData also) in values.yaml still getting below error.

2022/06/05 18:36:58 Loading config; env=docker,zone=,configDir=config
2022/06/05 18:36:58 Loading config files=[config/docker.yaml]
{"level":"info","ts":"2022-06-05T18:36:58.471Z","msg":"Build info","timestamp":"2021-12-13T20:31:36.000Z","git-revision":"074d8ebb1","platform":"amd64","go-version":"go1.17.3","server-version":"1.14.0","logging-call-at":"main.go:135"}
{"level":"info","ts":"2022-06-05T18:36:58.472Z","msg":"Updated dynamic config","logging-call-at":"file_based_client.go:142"}
2022/06/05 18:36:58 gocql: dns error: lookup []: no such host
Unable to start server. Error: could not build arguments for function "go.temporal.io/server/common/pprof".LifetimeHooks (/temporal/common/pprof/fx.go:39): failed to build *pprof.PProfInitializerImpl: could not build arguments for function "go.temporal.io/server/common/pprof".NewInitializer (/temporal/common/pprof/pprof.go:56): failed to build *config.PProf: could not build arguments for function "go.temporal.io/server/temporal".SoExpander (/temporal/temporal/fx.go:482): failed to build *temporal.serverOptions: received non-nil error from function "go.temporal.io/server/temporal".ServerOptionsProvider (/temporal/temporal/fx.go:508): cassandra schema version compatibility check failed: gocql: unable to create session: failed to resolve any of the provided hostnames

below are the details of helm values files:

project_name: "XXXXXX"

temporal:
  fullnameOverride: temporal
  web:
    enabled: true
    config:
      # server/config.yml file content
      auth:
        enabled: false
      routing:
        default_to_namespace: # internal use only
        issue_report_link: https://github.com/temporalio/web/issues/new/choose # set this field if you need to direct people to internal support forums


    replicaCount: 1

    image:
      repository: temporalio/web
      tag: 1.13.0
      pullPolicy: IfNotPresent

    service:
      type: LoadBalancer
      port: 8088
      annotations: {}
      # loadBalancerIP:

    ingress:
      enabled: false
      annotations: {}
      # kubernetes.io/ingress.class: traefik
      # ingress.kubernetes.io/ssl-redirect: "false"
      # traefik.frontend.rule.type: PathPrefix
      hosts:
        - "/"
        # - "domain.com/xyz"
        # - "domain.com"
      tls: []
      #  - secretName: chart-example-tls
      #    hosts:
      #      - chart-example.local

    podAnnotations: {}

    resources: {}
    # We usually recommend not to specify default resources and to leave this as a conscious
    # choice for the user. This also increases chances charts run on environments with little
    # resources, such as Minikube. If you do want to specify resources, uncomment the following
    # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
    # limits:
    #  cpu: 100m
    #  memory: 128Mi
    # requests:
    #  cpu: 100m
    #  memory: 128Mi

    nodeSelector: {}

    tolerations: []

    affinity: {}


  elasticsearch:
    enabled: false

  prometheus:
    enabled: false

  cassandra:
    enabled: false

  mysql:
    enabled: false

  server:
    image:
      repository: temporalio/server
      tag: 1.14.0
      pullPolicy: IfNotPresent
    config:
      numHistoryShards: 8
      persistence:
        default:
          driver: "cassandra"
          cassandra:
            driver: "cassandra"
            host: "34.XX.XXX.XX"
            port: 9042
            keyspace: temporal
            user: XXXXXX
            password: XXXXXXXX
            replicationFactor: 1
            maxConns: 200
            tls:
              caData: XXXXXXXXXXXXXXXXXXXXXXXXXXXXX.....XXX
              enabled: true
            consistency:
              default:
                consistency: "local_quorum"
                serialConsistency: "local_serial"

        visibility:
          driver: "cassandra"
          cassandra:
            driver: "cassandra"
            host: "34.XX.XXX.XXX"
            port: 9042
            keyspace: temporal_visibility
            user: XXXXXXX
            password: XXXXXXX
            replicationFactor: 1
            maxConns: 200
            tls:
              caData: XXXXXXXXXXXXXX...XXXXXX
              enabled: true
            consistency:
              default:
                consistency: "local_quorum"
                serialConsistency: "local_serial"
    frontend:
      # replicaCount: 1
      service:
        annotations: {} # Evaluated as template
        type: ClusterIP
        port: 7233
  admintools:
    enabled: true
    image:
      repository: temporalio/admin-tools
      tag: 1.14.0
      pullPolicy: IfNotPresent

  postgresql:
    enabled: false

  schema:
    setup:
      enabled: false
    update:
      enabled: false

grafana:
  enabled: false

dapr:
  enabled: false

otel:
  enabled: false

google-managed-prometheus:
  enabled: false

Is there anything missed?

I tried using docker comppose file also and trying to connect remote cassandra db, getting below error.

| + echo 'Waiting for Cassandra to start up.'
temporal              | Waiting for Cassandra to start up.
temporal              | + sleep 1
temporal              | + temporal-cassandra-tool --ep 35.194.6.202 validate-health
temporal              | 2022-06-06T07:53:53.782Z	e[34mINFOe[0m	Validating connection to cassandra cluster.	{"logging-call-at": "cqlclient.go:111"}
temporal              | 2022/06/06 07:53:54 gocql: unable to dial control conn 35.194.6.202:9042: x509: cannot validate certificate for 35.194.6.202 because it doesn't contain any IP SANs
temporal              | 2022-06-06T07:53:54.884Z	e[31mERRORe[0m	Connection validation failed.	{"error": "gocql: unable to create session: control: unable to connect to initial hosts: x509: cannot validate certificate for 35.194.6.202 because it doesn't contain any IP SANs", "logging-call-at": "cqlclient.go:114"}
temporal              | 2022-06-06T07:53:54.886Z	e[31mERRORe[0m	Unable to establish CQL session.	{"error": "gocql: unable to create session: control: unable to connect to initial hosts: x509: cannot validate certificate for 35.194.6.202 because it doesn't contain any IP SANs", "logging-call-at": "handler.go:137"}

A quick response would be highly appreciated.
Thanks in advance.
Regards
Prateek

Are you able to connect using cqlsh?

x509: cannot validate certificate for

Looks like issue with certificate creation, see here for some options on how to fix this.

I was able to connect using temporal Cassandra tool for creation/updation of schema using same certificate. But on docker compose getting above mentioned error.

1 Like