Temporal IO running activities of a workflow outside the parent workflow JAR

Community Support
,
1

I am using temporal SDKs with Java to run a temporal workflow, which has a few activities.
The last activity is to create and send a JMS message to a message queue for asynchronous processing. Since there are no steps after sending the JMS message, the temporal workflow is deemed to be completed. The process looks like this:

Temporal workflow -> Activity 1 -> Activity 2 -> Activity 3 (Send JMS message)

However, once the JMS message is received by the listener component, I want to continue the steps carried out in this listener component as activities of the same temporal workflow (which ended when the message was sent to the JMS queue).

How do I continue the activities of the existing workflow from the listener JMS component once the JMS message is sent from the temporal workflow?
The process that I require is:

Temporal workflow -> Activity 1 -> Activity 2 -> Activity 3 (Send JMS message)
Receive JMS message (in separate component/JAR) -> Activity 4 -> Activity 5

Reasons for using a JMS consumer for Activity 4 and 5 is to process these steps asynchronously and one at a time for multiple temporal workflow executions in the order JMS messages are received. Since the JMS consumer count is 1, a new message will not be consumed and processed until the activity 4 and 5 of the previous workflow execution are completed.

I have gone through the temporal documentation, but could not find anything to run Temporal activities outside of the code where the Temporal workflow is defined.

Stack overflow link for the same question: https://stackoverflow.com/questions/78472180/temporal-io-running-activities-of-a-workflow-outside-the-parent-workflow-jar

2

Why do you use JMS with Temporal? It is not needed, as Temporal supports asynchronous processing out of the box.

3

JMS is required for asynchronous running of activities which should run one at a time. Eg: There can be multiple workflow executions at a time, but the JMS consumer should execute the activity 4 and 5 for only one message at a time. Once the message is processed, the next message is received and the activity 4 and 5 are executed for the next workflow run.

4

You can configure a Temporal activity worker to process one activity at a time.

5

I am unable to find the docs for this configuration. Can you please share the docs link on how to implement in java? I can refer and try to implement, and see if it solves my use-case

6

WorkerOptions.MaxConcurrentActivityExecutionSize

7

Hi @maxim

Is there a way where I can add WorkerOptions.MaxConcurrentActivityExecutionSize only for Activity 4 and 5 (assuming all activities are running in same workflow)
I am using springboot-autoconfigure-alpha

8

Yes. You have to separate activities 4 and 5 into their own task queue and configure this limit in the worker that listens to that specific queue.

9

Awesome! Can you please provide some docs or some examples?

10

Got it. Ill try this and let you know.

11

Thanks @maxim, I was able to do that, one more question may be -
I have created this workflow method -

  @Override
  public void hitLongWorkflow() {
    logger.info("Starting long workflow");
    someActivity.doSomething1();
    someActivity.doSomething2();
    Promise<String> s = Async.function(nonBlockingActivity::doSomethingAsync);
    s.get();
  }

Is it possible that I dont wait for the completion of the nonBlockingActivity and just return, but without terminating the workflow. I tried removing s.get();, but the activity goes into pending becuase the workflow is marked as completed.

12

It is not possible. Activity cannot outlive the workflow that started it. What is the use case for this?

13

Hey
Use case -
There is a parent Workflow - which does 5 activities, the same as @shivam-lal told above.

Now, what I did - added all activity in a workflow, and the last 2 activity runs on a different TASK Queue with max-concurrent-activity-executor as 1.
What I want is - Once the last 2 activities run async (as in example) and the control comes out from the workflow method too.

Why do I want to close the Parent Workflow?
The project receives a JMS message (with max consumer = 1) and then initiates the Parent Workflow. The activity 4 and 5 take a bit of time to complete - 15-20 mins each. So I was thinking if there is a way, where I can come out of the workflow and pick the second message and initiate another parent workflow for the task received by MQ.

14

What is the reason for not keeping the parent workflow open until these two activities are complete?

15

added above, Thanks

16

I see. Is there a reason to execute the first part of the workflow one at a time? Can you return JMS listener after starting workflows?

Otherwise, you can use workflow Update to wait for workflow to reach a certain state.

17

Is there a reason to execute the first part of the workflow one at a time? Can you return JMS listener after starting workflows?

There are possibility where subsequent messages depends on the previous one. An example - first message do to a task, second message to update that task.

Workflow Update to wait for workflow to reach a certain state.

btw, what is this? Can you elaborate more?

18

There are possibility where subsequent messages depends on the previous one. An example - first message do to a task, second message to update that task.

Temporal will already buffer signals for you and deliver them in the same order they were received, so there is no need to try to manage concurrency externally. Consider using SignalWithStart to signal workflows; this would allow not thinking about which signals should start workflow and which should not.

You don’t need the update for this use case. But it is useful in many other scenarios: Temporal Application message passing - Signals, Queries, & Updates | Temporal Documentation

1 Like