Should I use workflow id or external datastore to keep state of previous messages in series or updates (revisions)

Community Support
1

Hi Team

Not sure of the best approach. I have a stream of Kafka events, multiple orders for multiple clients.
Each revision of the order has to been fully reprocessed in the workflow and generate a Kafka Message as output. I also need the state from any previous run as IDs are generated/incremented, reference data is looked up etc. I have tried using the externally generated order_number as the Workflow_id but hitting issues (WrokflowExecutionAlreadyStart). Is this the right approach and I’m just failing to understand something or should I have unique workflow ids and just lookup the previous revisions externally?

I’ve tried (And to find and example) in Java for something like this:

if Workflow Id {order_num} exists {
 Signal Workflow (or rerun)
 update state from old revision 
 publish Kafka event
} else {
  create new Workflow with order_num
  Run workflow
  publish Kafka Event
}

Any pointers most welcome
Thank you.

2

if Workflow Id {order_num} exists {
Signal Workflow (or rerun)
update state from old revision
publish Kafka event
} else {
create new Workflow with order_num
Run workflow
publish Kafka Event
}

you can use WorkflowStub.signalWithStart, for example:

WorkflowOptions workflowOptions =
                WorkflowOptions.newBuilder()
                        .setTaskQueue(TASK_QUEUE)
                        .setWorkflowId(orderNumber).build();

WorkflowStub workflow = client.newUntypedWorkflowStub("MyWorkflowType", workflowOptions);
workflow.signalWithStart("signalName", signalArgs, startArgs);

in case a workflow execution with the provided id does not exist, it will be created and signal is sent to it right away. If it exists and is running, then just the signal is delivered to it.

2 Likes