All pods are working ony worker is restarting. It’s fresh installation, empty database with new schema.
server-version":"1.16.1
"msg":"worker starting","service":"worker","component":"worker","logging-call-at":"service.go:329"}
{"level":"fatal","ts":"2022-05-05T09:13:40.649Z","msg":"error getting system sdk client","service":"worker","error":"unable to create SDK client: get system info failed: context deadline exceeded - *serviceerror.DeadlineExceeded","logging-call-at":"factory.go:98","stacktrace":"go.temporal.io/server/common/log.(*zapLogger).Fatal\n\t/home/builder/temporal/common/log/zap_logger.go:150\ngo.temporal.io/server/common/sdk.(*clientFactory).GetSystemClient.func1\n\t/home/builder/temporal/common/sdk/factory.go:98\nsync.(*Once).doSlow\n\t/usr/local/go/src/sync/once.go:68\nsync.(*Once).Do\n\t/usr/local/go/src/sync/once.go:59\ngo.temporal.io/server/common/sdk.(*clientFactory).GetSystemClient\n\t/home/builder/temporal/common/sdk/factory.go:94\ngo.temporal.io/server/service/worker.(*Service).startScanner\n\t/home/builder/temporal/service/worker/service.go:441\ngo.temporal.io/server/service/worker.(*Service).Start\n\t/home/builder/temporal/service/worker/service.go:355\ngo.temporal.io/server/service/worker.ServiceLifetimeHooks.func1.1\n\t/home/builder/temporal/service/worker/fx.go:129"}
unable to create SDK client: get system info failed: context deadline exceeded
Seems like issue setting up connection to start system workflows. Do you have TLS enabled, and can you see any connection errors in logs?
Are these errors transient (for example you only see them during redeploy or pod restart?) If so they can be ignored.
1 Like
Hi i am facing similar error while deploying temporal freshly , i am using latest helm repo version , any fix ?
{"level":"fatal","ts":"2022-05-24T11:29:03.054Z","msg":"error getting system sdk client","service":"worker","error":"unable to create SDK client: get system info failed: context deadline exceeded - *serviceerror.DeadlineExceeded","logging-call-at":"factory.go:98","stacktrace":"go.temporal.io/server/common/log.(*zapLogger).Fatal\n\t/home/builder/temporal/common/log/zap_logger.go:150\ngo.temporal.io/server/common/sdk.(*clientFactory).GetSystemClient.func1\n\t/home/builder/temporal/common/sdk/factory.go:98\nsync.(*Once).doSlow\n\t/usr/local/go/src/sync/once.go:68\nsync.(*Once).Do\n\t/usr/local/go/src/sync/once.go:59\ngo.temporal.io/server/common/sdk.(*clientFactory).GetSystemClient\n\t/home/builder/temporal/common/sdk/factory.go:94\ngo.temporal.io/server/service/worker.(*Service).startScanner\n\t/home/builder/temporal/service/worker/service.go:441\ngo.temporal.io/server/service/worker.(*Service).Start\n\t/home/builder/temporal/service/worker/service.go:355\ngo.temporal.io/server/service/worker.ServiceLifetimeHooks.func1.1\n\t/home/builder/temporal/service/worker/fx.go:129"}
This is the error I am able to see through worker pod’s logs
What do you get via
kubectl get po
Can you confirm that all your pods started up ?
Just tested with latest helm repo and it deployed fine (both k8s and minikube). This could me memory related as well.
All other pods frontend,history ,admintools are running fine , just the worker pod is crashing continuously. I dont think issue is with the memory since plenty of memory is vaialble
could you share details on which k8s version and temporal version you tried out ?
Hi, now I am facing the same issue, do you have any solution for this?
@Mundus_Creatus_Est, @swapnilk26 can you share how you solve this problem, thanks
Apologies to reply on old thread.
We are also facing same issue and worker service is not starting, can you please suggest what did you do to fix the issue?