After starting a workflow If I am restarting my service(which is hosting the workflow), the workflow is somehow getting stuck.
For ex →
WorkflowTaskTimedOut.
Make sure that your workflow implementation is deterministic. You can see associated rules here: Workflows in Java | Temporal documentation
Also make sure you look at your workflow exec history. There are many scenarios where WorkflowTaskTimeout can happen, see this post for more info, and looking at the history should provide info to find out the possible cause.
@tihomir It is not that every time I am getting this timeout. It is different every-time I am restarting the service. I am using the temporal-subscription-template only. The only difference is that in the sample the workflow is kicked by the main function itself. But in my case the work is started by an api.
@Pritam_Kumar2
So you are using GitHub - temporalio/subscription-workflow-project-template-java: Subscription workflow project template for Java
and your api is using the Java SDK client api to start the workflow exec? For restarting the service, is that random during workflow execution, and restarting your service takes down and brings back up all the workers? Trying to get as much info so can try to reproduce. What are you getting in your history log?
Yes My Api is using java SDK to start the execution.
I am not able to figure out how restart is taking down the workers but restart is not bringing up the workers. Worker start and registry is happening only through the API.
I think somehow the issue is that ki service restart is not doing workers restart and activities and workflow registry. Becuase when I declared a bean for that part so that at every restart Spring will run all the bean methods and workers will be restarted at every service restart, after that service restart was not affecting the workflow execution. But this raises few questions:
-
How and why exactly service restart is taking down the workers??
-
Do we need to start all my workers and and register all my activities and implementations at every service restart or production deployment(means inside some @Bean or @PostConstruct method)??
Workers are running inside your service. Stopping your service would likely stop the execution of the defined workers in that service. Don’t know how you are registering your workers inside your Spring app exactly, but idea is that you want to register them only once (on application start with ApplicationRunner or InitializingBean for example) and start your WorkerFactory. Similarly you would want to shutDown your WorkerFactory and your WorkflowClient gracefully on your application shutdown.
If in your service you are creating the worker processes then yes. Alternatively you could model your application such that only parts of your service can be restarted without bringing down your workers, and can update some other parts of it, but thats I think an architectural decision on how you do that.
Yeah I did the same, I put the following code in a @PostConstruct method:
WorkflowServiceStubs service = WorkflowServiceStubs.newInstance();
WorkflowClient client = WorkflowClient.newInstance(service, WorkflowClientOptions.newBuilder().setNamespace(“default”).build());
WorkerFactory factory = WorkerFactory.newInstance(client);
Worker worker = factor.newWorker(“taskQueue”);
factory.start();
Should I put the activity registration and implementation registration also in this method or I can do that on the fly when the “subscribe” api is called??