Local Cluster, external persistence

Community Support
1

I’d like to create a temporal cluster locally but spin up persistence in a hosted postgres instance. I’ve been hacking away with some of the docker-compose examples but keep running into tls issues when attempting to connect to the persistence layer.

Before I start another thread on the specific errors, is there a good guide for doing something like this?

2

What issues are you running into?
Here is the docker template for reference.
This forum post has a sample compose file with mysql running locally.
Would encourage you not to use auto-setup server image for anything but local testing. Here is a sample compose that deploys each server role in own container.

Hope this helps.

1 Like
3

Thank you for the reply! That does help.

Here’s what I’m trying to do:

I want to point my local cluster to a postgres instance hosted on render.com. I’ve been able to spin up a postgres instance and created the temporal, temporal_visibility dbs.

Here’s my docker-compose:

version: "3.5"
services:
  elasticsearch:
    container_name: temporal-elasticsearch
    environment:
      - cluster.routing.allocation.disk.threshold_enabled=true
      - cluster.routing.allocation.disk.watermark.low=512mb
      - cluster.routing.allocation.disk.watermark.high=256mb
      - cluster.routing.allocation.disk.watermark.flood_stage=128mb
      - discovery.type=single-node
      - ES_JAVA_OPTS=-Xms256m -Xmx256m
      - xpack.security.enabled=false
    image: elasticsearch:${ELASTICSEARCH_VERSION}
    networks:
      - temporal-network
    expose:
      - 9200
  temporal:
    container_name: temporal
    depends_on:
      - elasticsearch
    environment:
      - SQL_TLS=true
      - SQL_TLS_ENABLED=true
      - SQL_HOST_VERIFICATION=false
      - DYNAMIC_CONFIG_FILE_PATH=config/dynamicconfig/development-sql.yaml
      - DB=postgresql
      - SKIP_DB_CREATE=true
      - POSTGRES_USER=<USER>
      - POSTGRES_PWD=<PWD>
      - DB_PORT=5432
      - POSTGRES_SEEDS=<POSTGRES_HOST>
      - ENABLE_ES=true
      - ES_SEEDS=elasticsearch
      - ES_VERSION=v7
    image: temporalio/auto-setup:${TEMPORAL_VERSION}
    networks:
      - temporal-network
    ports:
      - 7233:7233
    labels:
      kompose.volume.type: configMap
    volumes:
      - ./dynamicconfig:/etc/temporal/config/dynamicconfig
  temporal-admin-tools:
    container_name: temporal-admin-tools
    depends_on:
      - temporal
    environment:
      - TEMPORAL_CLI_ADDRESS=temporal:7233
    image: temporalio/admin-tools:${TEMPORAL_VERSION}
    networks:
      - temporal-network
    stdin_open: true
    tty: true
  temporal-ui:
    container_name: temporal-ui
    depends_on:
      - temporal
    environment:
      - TEMPORAL_ADDRESS=temporal:7233
      - TEMPORAL_CORS_ORIGINS=http://localhost:3000
    image: temporalio/ui:${TEMPORAL_UI_VERSION}
    networks:
      - temporal-network
    ports:
      - 8080:8080
  temporal-web:
    container_name: temporal-web
    depends_on:
      - temporal
    environment:
      - TEMPORAL_GRPC_ENDPOINT=temporal:7233
      - TEMPORAL_PERMIT_WRITE_API=true
    image: temporalio/web:${TEMPORAL_WEB_VERSION}
    networks:
      - temporal-network
    ports:
      - 8088:8088
networks:
  temporal-network:
    driver: bridge
    name: temporal-network

I can see the auto-setup logs from this script. I can see that it correctly applies the schemas and verifies them. That it’s able to connect to my hosted db.

It’s failing to start with this error:
Unable to start server. Error: could not build arguments for function "go.temporal.io/server/common/pprof".LifetimeHooks (/home/builder/temporal/common/pprof/fx.go:39): failed to build *pprof.PProfInitializerImpl: could not build arguments for function "go.temporal.io/server/common/pprof".NewInitializer (/home/builder/temporal/common/pprof/pprof.go:56): failed to build *config.PProf: received non-nil error from function "go.temporal.io/server/temporal".ServerOptionsProvider (/home/builder/temporal/temporal/fx.go:152): sql schema version compatibility check failed: EOF

I’ve tried digging into the source code but cannot find where it could fail with an EOF. I suspect it has something to do with the TLS settings (and not providing certs).

Any pointers would be deeply appreciated.

4

I was able to resolve it. Basically I had to setup a VPN to hook up my local cluster to my hosted db.

I had to setup a tailscale subnet router for render.

Then I had to set
SQL_HOST_VERIFICATION=true
SQL_CA=<path to ca for tailscale (using a tailscale subnet router and configuring certs for my local machine)>
SQL_CERT=<path to the cert for my local machine (generated from tailscale)>
SQL_CERT_KEY=<path to the cert for my local machine (generated from tailscale)>

Then I was able to hook up my local cluster to a db in render. Thank you for the help!

1 Like
5

Actually this just needs to be set to true. Tailscale is not required.