Running into an issue when creating namespace programmatically

I am creating namespace programmatically

       Worker workflowWorker = workerFactory.newWorker(WORKFLOW_TASK_QUEUE);
        workflowWorker.registerWorkflowImplementationTypes(WorkflowImpl.class);
        workflowWorker.registerActivitiesImplementations(activity);
        workerFactory.start();

        /**
         * Register namespace if not exists on the temporal server. This logic is behind a feature flag.
         */
        final boolean autoRegisterNamespace =
            (boolean)environment.getProperty("temporal.namespace.autoregister", Boolean.class, DEFAULT_NAMESPACE_AUTOREGISTER); // registration behind boolean flag
        if (autoRegisterNamespace) {
            RegisterNamespaceRequest request = RegisterNamespaceRequest.newBuilder()
                                                                   .setNamespace("hello-world")
                                                                   .setHistoryArchivalState(enableArchival
                                                                                   ? ArchivalState.ARCHIVAL_STATE_ENABLED
                                                                                   : ArchivalState.ARCHIVAL_STATE_DISABLED)
                                                                   .setDescription(NAMESPACE_DESCRIPTION)
                                                                   .setVisibilityArchivalState(enableArchival
                                                                                   ? ArchivalState.ARCHIVAL_STATE_ENABLED
                                                                                   : ArchivalState.ARCHIVAL_STATE_DISABLED)
                                                                   .setWorkflowExecutionRetentionPeriod(
                                                                                                        Durations.fromDays(workflowRetentionDays))
                                                                   .build();
        }

However I am getting this exception
io.grpc.StatusRuntimeException: NOT_FOUND: namespace: hello-world not found

Stacktrace

{"@timestamp":"2021-08-24T21:26:00.920+00:00","@version":"1","message":"Failure in thread Host Local Workflow Poller: 3","

logger_name":"io.temporal.internal.worker.Poller","thread_name":"Host Local Workflow Poller: 3","level":"ERROR","level_val

ue":40000,"stack_trace":"io.grpc.StatusRuntimeException: NOT_FOUND: namespace: hello-world not found\n\tat io.grpc.stub.ClientC

alls.toStatusRuntimeException(ClientCalls.java:262)\n\tat io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:243)\n\ta

t io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:156)\n\tat io.temporal.api.workflowservice.v1.WorkflowServic

eGrpc$WorkflowServiceBlockingStub.pollWorkflowTaskQueue(WorkflowServiceGrpc.java:2639)\n\tat io.temporal.internal.worker.W

orkflowPollTask.poll(WorkflowPollTask.java:81)\n\tat io.temporal.internal.worker.WorkflowPollTask.poll(WorkflowPollTask.ja

va:37)\n\tat io.temporal.internal.worker.Poller$PollExecutionTask.run(Poller.java:270)\n\tat io.temporal.internal.worker.P

oller$PollLoopTask.run(Poller.java:235)\n\tat datadog.trace.bootstrap.instrumentation.java.concurrent.Wrapper.run(Wrapper.

java:25)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base/j

ava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base/java.lang.Thread.run(Thread

.java:829)\n"}

I wonder if this is due to the fact that the activities and workflow are registered prior to creating the namespace. If yes, when should one ideally call the logic to create namespace programmatically ?

You usually create a namespace from a separate process or tctl before starting your workers.

But I believe that the error you posted should go away after 10 seconds which is the time it takes for the namespace cache to be updated.

Strangeā€¦ I saw this exception for a couple of hours on the temporal instance. @maxim

Did restarting workers help? Did you see the namespace in the tctl namespace describe?