# Adding a cluster using dns fails

**URL:** <https://community.temporal.io/t/adding-a-cluster-using-dns-fails/4150>\
**Category:** Community Support\
**Tags:** multicluster, deployment, tctl\
**Created:** [March 10, 2022, 4:31pm UTC](https://community.temporal.io/t/adding-a-cluster-using-dns-fails/4150 "2022-03-10T16:31:31Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![madhu](https://avatars.discourse-cdn.com/v4/letter/m/a8b319/32.png) [@madhu](https://community.temporal.io/u/madhu)\
**Post date:** [March 10, 2022, 4:31pm UTC](https://community.temporal.io/t/adding-a-cluster-using-dns-fails/4150/1 "2022-03-10T16:31:31Z")

</div>

tctl -ns mynamespace adm cl urc --frontend\_address dns:///temporal-xdc-active.mydomain:7233

fails with Error: Operation AddOrUpdateRemoteCluster failed.  
Error Details: rpc error: code = Unavailable desc = error reading from server: read tcp 10.x.yyy.zzz:36808-\>172.xx.yyy.zz:7233: read: connection reset by peer

and i see error in worker  
{“level”:“warn”,“ts”:“2022-03-10T16:28:55.362Z”,“msg”:“Failed to poll for task.”,“Namespace”:“temporal-system”,“TaskQueue”:“temporal-sys-tq-scanner-taskqueue-0”,“WorkerID”:“1@myserver-temporal-worker-7b6b74cf66-h7gdc@”,“WorkerType”:“ActivityWorker”,“Error”:“error reading from server: EOF”,“logging-call-at”:“internal\_worker\_base.go:276”}

Where as tctl -ns mynamspace adm cl urc --frontend\_address 10.x.yyy.zzz:7233 suceeds

any idea whats going wrong? i suspect there after upgrading to 1.15.2 the dns:/// possibly broke

---

<div class="post-metadata">

**Author:** ![yux](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/yux/32/1625_2.png) [@yux](https://community.temporal.io/u/yux)\
**Post date:** [March 10, 2022, 6:05pm UTC](https://community.temporal.io/t/adding-a-cluster-using-dns-fails/4150/2 "2022-03-10T18:05:18Z")

</div>

We do not change the AddOrUpdateRemoteCluster in v1.15. From the logging mesasge, the valid address is 10.x.yyy.zzz:7233. When using dns, does it forward the request (10.x.yyy.zzz:36808-\>172.xx.yyy.zz:7233)? should the port be 7233?

---

<div class="post-metadata">

**Author:** ![madhu](https://avatars.discourse-cdn.com/v4/letter/m/a8b319/32.png) [@madhu](https://community.temporal.io/u/madhu)\
**Post date:** [March 11, 2022, 4:54am UTC](https://community.temporal.io/t/adding-a-cluster-using-dns-fails/4150/3 "2022-03-11T04:54:06Z")

</div>

> [@yux](#):
>
> 233)? should the port be 7233?

yes 7233 is part of the url and it works pefectly fine in 1.14.x and 1.15.0 ( i upgraded from 1.14.3 to 1.15.0)

---

<div class="post-metadata">

**Author:** ![madhu](https://avatars.discourse-cdn.com/v4/letter/m/a8b319/32.png) [@madhu](https://community.temporal.io/u/madhu)\
**Post date:** [March 17, 2022, 4:11pm UTC](https://community.temporal.io/t/adding-a-cluster-using-dns-fails/4150/4 "2022-03-17T16:11:52Z")

</div>

i tested this again. i am consistently reproduce this.

tctl -nsmynamespace adm cl urc --fad dns:///mycluster:7233  
Error: Operation AddOrUpdateRemoteCluster failed.  
Error Details: rpc error: code = Unavailable desc = error reading from server: EOF

And when this happens i see this error in my front end

```auto
{"level":"fatal","ts":"2022-03-17T16:08:52.595Z","msg":"Invalid rpcAddress for remote cluster","error":"address dns:///myaddress:7233: too many colons in address","logging-call-at":"rpc.go:268","stacktrace":"go.temporal.io/server/common/log.(*zapLogger).Fatal\n\t/temporal/common/log/zap_logger.go:150\ngo.temporal.io/server/common/rpc.(*RPCFactory).CreateFrontendGRPCConnection\n\t/temporal/common/rpc/rpc.go:268\ngo.temporal.io/server/client.(*rpcClientFactory).NewAdminClientWithTimeout.func1\n\t/temporal/client/clientfactory.go:213\ngo.temporal.io/server/common.(*clientCacheImpl).GetClientForClientKey\n\t/temporal/common/clientCache.go:94\ngo.temporal.io/server/common.(*clientCacheImpl).GetClientForKey\n\t/temporal/common/clientCache.go:75\ngo.temporal.io/server/client/admin.(*clientImpl).getRandomClient\n\t/temporal/client/admin/client.go:473\ngo.temporal.io/server/client/admin.(*clientImpl).DescribeCluster\n\t/temporal/client/admin/client.go:253\ngo.temporal.io/server/client/admin.(*metricClient).DescribeCluster\n\t/temporal/client/admin/metricClient.go:294\ngo.temporal.io/server/service/frontend.(*AdminHandler).AddOrUpdateRemoteCluster\n\t/temporal/service/frontend/adminHandler.go:1055\ngo.temporal.io/server/api/adminservice/v1._AdminService_AddOrUpdateRemoteCluster_Handler.func1\n\t/temporal/api/adminservice/v1/service.pb.go:968\ngo.temporal.io/server/common/rpc/interceptor.(*SDKVersionInterceptor).Intercept\n\t/temporal/common/rpc/interceptor/sdk_version.go:63\ngoogle.golang.org/grpc.chainUnaryInterceptors.func1.1\n\t/go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1116\ngo.temporal.io/server/common/authorization.(*interceptor).Interceptor\n\t/temporal/common/authorization/interceptor.go:152\ngoogle.golang.org/grpc.chainUnaryInterceptors.func1.1\n\t/go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1119\ngo.temporal.io/server/common/rpc/interceptor.(*NamespaceCountLimitInterceptor).Intercept\n\t/temporal/common/rpc/interceptor/namespace_count_limit.go:99\ngoogle.golang.org/grpc.chainUnaryInterceptors.func1.1\n\t/go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1119\ngo.temporal.io/server/common/rpc/interceptor.(*NamespaceRateLimitInterceptor).Intercept\n\t/temporal/common/rpc/interceptor/namespace_rate_limit.go:89\ngoogle.golang.org/grpc.chainUnaryInterceptors.func1.1\n\t/go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1119\ngo.temporal.io/server/common/rpc/interceptor.(*RateLimitInterceptor).Intercept\n\t/temporal/common/rpc/interceptor/rate_limit.go:84\ngoogle.golang.org/grpc.chainUnaryInterceptors.func1.1\n\t/go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1119\ngo.temporal.io/server/common/rpc/interceptor.(*NamespaceValidatorInterceptor).Intercept\n\t/temporal/common/rpc/interceptor/namespace_validator.go:113\ngoogle.golang.org/grpc.chainUnaryInterceptors.func1.1\n\t/go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1119\ngo.temporal.io/server/common/rpc/interceptor.(*TelemetryInterceptor).Intercept\n\t/temporal/common/rpc/interceptor/telemetry.go:108\ngoogle.golang.org/grpc.chainUnaryInterceptors.func1.1\n\t/go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1119\ngo.temporal.io/server/common/metrics.NewServerMetricsContextInjectorInterceptor.func1\n\t/temporal/common/metrics/grpc.go:66\ngoogle.golang.org/grpc.chainUnaryInterceptors.func1.1\n\t/go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1119\ngo.temporal.io/server/common/rpc.ServiceErrorInterceptor\n\t/temporal/common/rpc/grpc.go:131\ngoogle.golang.org/grpc.chainUnaryInterceptors.func1.1\n\t/go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1119\ngo.temporal.io/server/common/rpc/interceptor.(*NamespaceLogInterceptor).Intercept\n\t/temporal/common/rpc/interceptor/namespace_logger.go:84\ngoogle.golang.org/grpc.chainUnaryInterceptors.func1.1\n\t/go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1119\ngoogle.golang.org/grpc.chainUnaryInterceptors.func1\n\t/go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1121\ngo.temporal.io/server/api/adminservice/v1._AdminService_AddOrUpdateRemoteCluster_Handler\n\t/temporal/api/adminservice/v1/service.pb.go:970\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1282\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:1616\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.44.0/server.go:921"}

```

---

<div class="post-metadata">

**Author:** ![madhu](https://avatars.discourse-cdn.com/v4/letter/m/a8b319/32.png) [@madhu](https://community.temporal.io/u/madhu)\
**Post date:** [March 17, 2022, 4:46pm UTC](https://community.temporal.io/t/adding-a-cluster-using-dns-fails/4150/5 "2022-03-17T16:46:36Z")

</div>

could this commit have broken the front end url parsing logic [Add config and handling for remote cluster cert (#2475) · temporalio/temporal@305a6fc · GitHub](https://github.com/temporalio/temporal/commit/305a6fc97e11d754c7eb77afa406d6626c64216e)

look at this test cases specfically which add “:” while parsing port [temporal/rpc\_common\_test.go at 305a6fc97e11d754c7eb77afa406d6626c64216e · temporalio/temporal · GitHub](https://github.com/temporalio/temporal/blob/305a6fc97e11d754c7eb77afa406d6626c64216e/common/rpc/test/rpc_common_test.go#L86-L90)

---

<div class="post-metadata">

**Author:** ![yux](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/yux/32/1625_2.png) [@yux](https://community.temporal.io/u/yux)\
**Post date:** [March 22, 2022, 8:55pm UTC](https://community.temporal.io/t/adding-a-cluster-using-dns-fails/4150/6 "2022-03-22T20:55:50Z")

</div>

> [@madhu](#):
>
> `too many colons in address`

The error indicate invalid address. could you try address:7233? refer [tls error](https://stackoverflow.com/questions/50347441/tls-dial-returns-too-many-colons-in-address)

---

<div class="post-metadata">

**Author:** ![madhu](https://avatars.discourse-cdn.com/v4/letter/m/a8b319/32.png) [@madhu](https://community.temporal.io/u/madhu)\
**Post date:** [March 23, 2022, 5:40am UTC](https://community.temporal.io/t/adding-a-cluster-using-dns-fails/4150/7 "2022-03-23T05:40:28Z")

</div>

Hmm. could be… i think this started breaking after 1.15.x upgrade(after the grpc library upgrade in temporal), this was working fine till 1.14.x

Unfortunately, I will need the port to be mentioned because the dns address i am trying to resovle belongs. let me see if there is work around to it.

---

<div class="post-metadata">

**Author:** ![meiliang86](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/meiliang86/32/1504_2.png) [@meiliang86](https://community.temporal.io/u/meiliang86)\
**Post date:** [March 23, 2022, 9:02pm UTC](https://community.temporal.io/t/adding-a-cluster-using-dns-fails/4150/8 "2022-03-23T21:02:14Z")

</div>

> [@madhu](#):
>
> tctl -nsmynamespace adm cl urc --fad dns:///mycluster:7233

have you tried to add cluster using just cluster\_name:port?

---

<div class="post-metadata">

**Author:** ![madhu](https://avatars.discourse-cdn.com/v4/letter/m/a8b319/32.png) [@madhu](https://community.temporal.io/u/madhu)\
**Post date:** [April 7, 2022, 4:49pm UTC](https://community.temporal.io/t/adding-a-cluster-using-dns-fails/4150/9 "2022-04-07T16:49:22Z")

</div>

> [@meiliang86](#):
>
> using just cluster\_name:port

Sorrry for delayed reply, i was out of office.  
Yes, i tried it today, and unfortunately that did not work either, using ip address works fine.dns:///:port was working fine in 1.4.x

---

<div class="post-metadata">

**Author:** ![madhu](https://avatars.discourse-cdn.com/v4/letter/m/a8b319/32.png) [@madhu](https://community.temporal.io/u/madhu)\
**Post date:** [April 9, 2022, 7:10am UTC](https://community.temporal.io/t/adding-a-cluster-using-dns-fails/4150/10 "2022-04-09T07:10:09Z")

</div>

i realized that url without dns:/// works, but, there are couple of thing which i had to do before making it work.

a) i had to delete clustermetadata and clustermetadatainfo table.  
b) and i had to to promote my namespace again using tctl -ns mynamespace n up --pn
