Execute workflow across workers

Community Support
1

Hello,

In one of our integration tests I’m starting a Temporal workflow and scheduling some work, then subsequently killing the worker by calling shutdown() on the worker factory.

I’m then creating a new worker of the same workflow type, and would like for it to pick up work on the Temporal workflow that was started on the previous worker from where it left off. I’d also like to get a stub referencing the workflow so I can query the workflow.

How can I do this?

2

You don’t need to do anything special. Just bring the new worker and it will pick up the workflow where it is left off.

3

I bring up the new worker, as confirmed by the logs:

23-04-2025 09:39:02.816 [Test worker] INFO  io.temporal.internal.worker.Poller.start - start: Poller{name=Workflow Poller taskQueue="test-books-task-queue", namespace="test-core-contract", identity=65937@SC00CGSn383453M}
23-04-2025 09:39:02.817 [Test worker] INFO  io.temporal.internal.worker.Poller.start - start: Poller{name=Activity Poller taskQueue="test-books-task-queue", namespace="test-core-contract", identity=65937@SC00CGSn383453M}

But I’m not able to query the workflow. I create a new stub using workflowClient.newWorkflowStub(workflowInterface, workflowId) but attempts to query it result in a timeout:

workflowId='B0000000000000001166', runId='', workflowType='BookWorkflow'
io.temporal.client.WorkflowServiceException: workflowId='B0000000000000001166', runId='', workflowType='BookWorkflow'
	at io.temporal.client.WorkflowStubImpl.throwAsWorkflowFailureExceptionForQuery(WorkflowStubImpl.java:520)
	at io.temporal.client.WorkflowStubImpl.query(WorkflowStubImpl.java:321)
	at io.temporal.testing.TimeLockingInterceptor$TimeLockingWorkflowStub.query(TimeLockingInterceptor.java:173)
	at io.temporal.client.WorkflowInvocationHandler$SyncWorkflowInvocationHandler.queryWorkflow(WorkflowInvocationHandler.java:319)
	at io.temporal.client.WorkflowInvocationHandler$SyncWorkflowInvocationHandler.invoke(WorkflowInvocationHandler.java:280)
	at io.temporal.client.WorkflowInvocationHandler.invoke(WorkflowInvocationHandler.java:186)
	at jdk.proxy3/jdk.proxy3.$Proxy355.getState(Unknown Source)
	at com.santander.core.contract.service.orchestration.AbstractDomainObjectWorkflowIT.do something(AbstractDomainObjectWorkflowIT.kt:138)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
Caused by: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: query deadline exceeded
	at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:271)
	at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:252)
	at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:165)
	at io.temporal.api.workflowservice.v1.WorkflowServiceGrpc$WorkflowServiceBlockingStub.queryWorkflow(WorkflowServiceGrpc.java:4821)
	at io.temporal.internal.client.external.GenericWorkflowClientImpl.lambda$query$10(GenericWorkflowClientImpl.java:208)
	at io.temporal.internal.retryer.GrpcSyncRetryer.retry(GrpcSyncRetryer.java:69)
	at io.temporal.internal.retryer.GrpcRetryer.retryWithResult(GrpcRetryer.java:60)
	at io.temporal.internal.client.external.GenericWorkflowClientImpl.query(GenericWorkflowClientImpl.java:203)
	at io.temporal.internal.client.RootWorkflowClientInvoker.query(RootWorkflowClientInvoker.java:424)
	at io.temporal.client.WorkflowStubImpl.query(WorkflowStubImpl.java:317)
	... 9 more
4

This seems to be happening only with the test Temporal server created using TestWorkflowEnvironment.newInstance(). If I use a remote Temporal server, i.e. one running on Docker, then I can query the workflow as expected.

So somehow shutting down the workers is also impacting the test Temporal instance.

5

Are you creating the new workflow from testEnv? Which task queue are you specifying for the worker and are you shutting down / realcreating test environment between tests?

6

Are you creating the new workflow from testEnv?

Yes, the workflow is created from a workflow client obtained from the testEnv

Which task queue are you specifying for the worker?

Using the same task queue for both workers.

Code looks kind of like this:

val firstFactory = WorkerFactory(workflowClient)
firstFactory.newWorker(taskQueueName)
firstFactory.start()
// ... create workflow
firstFactory.shutdown() // kill all existing workers

val newFactory = WorkerFactory(workflowClient)
newFactory.newWorker(taskQueueName)
newFactory.start()
// ... query workflow

are you shutting down / realcreating test environment between tests?

The issue occurs within a single test execution. I’m not shutting down the test env nor restarting it, just the worker factory.