Not able to register namespace in 1.2.1 k8s setup

Community Support
1

Hi i have been running 1.2.1 docker image for couple of days and it works perfectly fine.

When i try to run temporal 1.2.1 using k8s/helm.

I am unable to register namespace, i get 403 exception.
bash-5.0# tctl --namespace samples-namespace namespace register
Error: Register namespace operation failed.
Error Details: rpc error: code = Unknown desc = Forbidden: Forbidden
status code: 403, request id: 250DCE81B9026FA0, host id: bTkqpS/CChOZrwZw14CL1p8mOfKlnEMU66Pm25Mx33DxGk6a+KD6xLvdso3cX7xhjAopvQ/dUGA=
(‘export TEMPORAL_CLI_SHOW_STACKS=1’ to see stack traces)

in the front end i see the following in logs

{“level”:“error”,“ts”:“2020-11-11T11:22:30.917Z”,“msg”:“Unknown error”,“service”:“frontend”,“error”:“Forbidden: Forbidden\n\tstatus code: 403, request id: C486AFCD8246BCA6, host id: p3YC1NdyUDJbVfGd6OrQ8IYuLcOmqSYpQaauroLUyQUoF7qkekVFxR+mQrX5bgQC3DVf2/L7BWI=”,“logging-call-at”:“workflowHandler.go:3399”,“stacktrace”:“go.temporal.io/server/common/log/loggerimpl.(*loggerImpl).Error\n\t/temporal/common/log/loggerimpl/logger.go:138\ngo.temporal.io/server/service/frontend.(*WorkflowHandler).error\n\t/temporal/service/frontend/workflowHandler.go:3399\ngo.temporal.io/server/service/frontend.(*WorkflowHandler).RegisterNamespace\n\t/temporal/service/frontend/workflowHandler.go:252\ngo.temporal.io/server/service/frontend.(*DCRedirectionHandlerImpl).RegisterNamespace\n\t/temporal/service/frontend/dcRedirectionHandler.go:177\ngo.temporal.io/server/service/frontend.(*AccessControlledWorkflowHandler).RegisterNamespace\n\t/temporal/service/frontend/accessControlledHandler.go:469\ngo.temporal.io/server/service/frontend.(*WorkflowNilCheckHandler).RegisterNamespace\n\t/temporal/service/frontend/workflowNilCheckHandler.go:62\ngo.temporal.io/api/workflowservice/v1._WorkflowService_RegisterNamespace_Handler.func1\n\t/go/pkg/mod/go.temporal.io/api@v1.1.0/workflowservice/v1/service.pb.go:957\ngo.temporal.io/server/common/rpc.Interceptor\n\t/temporal/common/rpc/interceptor.go:35\ngo.temporal.io/api/workflowservice/v1._WorkflowService_RegisterNamespace_Handler\n\t/go/pkg/mod/go.temporal.io/api@v1.1.0/workflowservice/v1/service.pb.go:959\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.32.0/server.go:1194\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.32.0/server.go:1517\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.32.0/server.go:859”}

2

ok. looks like it was EKS s3 archival credentails issue.
will be nice we there could be some hint in the logs that its coming from archival configuration.

3

Would you file an issue to improve the logging?

4

Thank @maxim just did that https://github.com/temporalio/temporal/issues/983

1 Like