Invoke workflow from rest endpoint

[carrotcap3]

I plan to invoke temporal workflow that can take a few minutes to complete from a rest api endpoint. i.e. the api POST /invoke/temporal/workflow will invoke a temporal workflow that will do processing. My question is do I use async to invoke the workflow, because I want to send a response back to the client instead of blocking for many minutes.

i.e.
@PostMapping("/invoke/temporal/workflow")
public Response startWorkflow() {
Promise hello = Async.function(activities::composeGreeting, “Hello”, name);
return http 200;
}

Once this line is executed, temporal will store the workflow and retries will automatically be done upon failure, is my assumption correct?

[maxim]

Async class should be used only from the workflow code. Inside a rest handler use WorkflowClient to start workflow without waiting for it to complete:

      MyWorkflow workflow =
          client.newWorkflowStub(MyWorkflow.class, workflowOptions);

      // Start asynchronously
      WorkflowExecution execution =
          WorkflowClient.start(workflow::execute, arg1);

[carrotcap3]

Thanks for this. I am trying to test the activity retry capabilities but am running into RETRY_STATE_TIMEOUT on what looks like the workflow after about 5-6 minutes of the activity attempting a retries. I configured the activity to do many retries, for many days but yet the workflow always times out after a short time and I am unsure why.

15:32:53.871 WARN  i.t.i.s.POJOActivityTaskHandler - Activity failure. ActivityId=b26d696c-785a-39cd-9c82-d6a376575fea, activityType=Process, attempt=6 
java.lang.NullPointerException: npe so should do retries
	at temporal.activity.MyFunActivityImpl.process(MyFunActivityImpl.java:36)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at io.temporal.internal.sync.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:286)
	at io.temporal.internal.sync.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:252)
	at io.temporal.internal.sync.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:209)

5:32:53.905 ERROR i.t.i.s.WorkflowExecuteRunnable - Workflow execution failure WorkflowId=MyFunWorkflow2021-11-25T15:27:12], RunId=5cc2195f-cd5b-43f8-986b-7ea4f2ee52aa, WorkflowType=MyFunWorkflow 
io.temporal.failure.ActivityFailure: scheduledEventId=5, startedEventId=6, activityType='Process', activityId='b26d696c-785a-39cd-9c82-d6a376575fea', identity='27315@ABC21342', retryState=RETRY_STATE_TIMEOUT
	at java.base/java.lang.Thread.getStackTrace(Thread.java:1607)
	at io.temporal.internal.sync.ActivityStubBase.execute(ActivityStubBase.java:48)
	at io.temporal.internal.sync.ActivityInvocationHandler.lambda$getActivityFunc$0(ActivityInvocationHandler.java:77)
	at io.temporal.internal.sync.ActivityInvocationHandlerBase.invoke(ActivityInvocationHandlerBase.java:70)
	at com.sun.proxy.$Proxy431.process(Unknown Source)
	at temporal.workflow.MyFunWorkflow.runWorkflow(MyFunWorkflow.java:41)
Caused by: io.temporal.failure.ApplicationFailure: message='npe so should do retries', type='java.lang.NullPointerException', nonRetryable=false
	at temporal.activity.MyFunActivity.process(MyFunActivity.java:36)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method:0)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at io.temporal.internal.sync.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:286)
	at io.temporal.internal.sync.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:252)
	at io.temporal.internal.sync.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:209)
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:193)
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:151)

[tihomir]

Can you show your workflow history? Also your WorkflowOptions and ActivityOptions.

[carrotcap3]

Using default workflow options

[carrotcap3]

[maxim]

You configured ScheduleToClose timeout of 16m 40s. This timeout limits the duration of retries.

I recommend watching The 4 Types of Activity timeouts video/post that explains each activity timeout in detail.

[carrotcap3]

I"ll take a look thank you. Does this mean temporal keeps the workflow “running” on a thread while these retries keep waiting for potential hours or days? That seems to limit the amount of workflow instances I can have that are in a retry state if they are always running. I thought that the workflow gets suspended or queued if its waiting for activity retry scheduled time.

I guess my question is, is there a better way to setup temporal for the scenario where I have a 1 to 1 mapping of activity and workflow, and activity can fail and is retriable for a few days

[maxim]

Does this mean temporal keeps the workflow “running” on a thread while these retries keep waiting for potential hours or days?

No, it doesn’t mean this. Suspended workflow doesn’t take any resources from the worker.

I guess my question is, is there a better way to setup temporal for the scenario where I have a 1 to 1 mapping of activity and workflow, and activity can fail and is retriable for a few days

Yes, don’t set any timeouts besides activity StartToClose timeout that limits the duration of a single activity execution attempt. Also don’t set RetryPolicy.MaximumAttempts as it practically never has any business related value.