We have a use-case where a micro-service (e.g. order management) should first persist the order in its own data base and trigger a fulfillment workflow (using Temporal). We want to make sure that overall the process is transactionally consistent. E.g. if the save of the order fails the workflow should not be started and vice-versa.
There are some patterns like the outbox pattern that is used in other such scenarios (like publishing an event to Kafka). Wanted to understand if there is a recommended pattern that we should be using with Temporal.
The recommended pattern is to use workflow as the source of truth while the workflow is open. In this case, you trigger the workflow and let the workflow update DB through an activity.
All queries and future updates (in form of signals) are against the workflow which is fully consistent. So no inconsistency is present. Before completing the workflow the DB is updated with the latest state and can be used for historical reasons.
Thanks @maxim ! Overall from a consistency PoV, this approach makes sense. One trade-off with this approach, IMHO, is that the complete Entity/Aggregate model payload would need to be passed to the workflow. Whereas if you use something like the outbox pattern the workflow can just be based on the persisted entity (just a key / id).
I’m not sure why you need to pass the whole entity to the workflow. In most cases, the workflow state itself is such an entity and is built step by step during workflow execution.
Interesting. What about checking the validity of the trigger request synchronously?
Suppose I implement
create <id> API where the id must be unique within a customer account.
My API gateway could atomically check and insert this id into the database as part of the API call and notify the user if something is wrong. It seems I can not do it if I move this part into the asynchronous workflow.
Temporal provides id uniqueness guarantees. So it is not possible to have two open workflows that share the same id. So your use case is trivially implementable with Temporal.