I am new to using temporal and gone through couple of articles but were not able to find desired answer.
Let’s take below example
Create product using REST API and persist to DB, respond with product created immediately
Sync above created product data asynchronously to other 2 external services (We can add more services later on to sync same data)
Questions
How can we guarantee using temporal that product is synced to all the services eventually.
If one or more service fails (due to network or bug) to sync this data to external service then how can we guarantee that, it will retry again in future (say after we fix the bug in syncing logic)
What I want that eventually product state should be same in all the services as in DB.
In traditional way we can use outbox pattern with kafka and debezium. Until we consume kafka event and ack service is going to retry to sync it to external service.
Other approach might be to have scheduler which keeps track of product version and make sure same version is present in all the services (this can be done by keeping track of updated_at in product table and say integration table where we keep track of updated_at for all services) and if not then try to sync it again.
Wondering how this can be achieved using temporal, as I have seen in couple of blogs that kafka is not needed if we are using temporal
How can we guarantee using temporal that product is synced to all the services eventually.
Temporal guarantees that the workflow function is going to execute eventually. So in your case it would mean that all products will be synced.
If one or more service fails (due to network or bug) to sync this data to external service then how can we guarantee that, it will retry again in future (say after we fix the bug in syncing logic)
There are many ways to solve this. The most simple one is to keep retrying the operation (using exponential backoff) until the fix is deployed.
@maxim Thanks for your reply. This is helpful. One more question, should I create a workflow per REST API and spawn child workflows for each service from parent workflow?
Approach 1
Create product Parent workflow
Save product to db (using activity)
spawn child workflow to sync this product with service 1
spawn child workflow to sync this product with service 2
… etc.
Approach 2
Create product workflow
workflow method to save to db
workflow method to sync to service 1
workflow method to sync to service 2
Which approach should I use to best handle this case?
You don’t need child workflows unless each product synching is a complex flow owned by another team.
I would go with approach 2. BTW you can use the full power of the language when implementing workflows. So the synching can leave in its own class, not necessarily the workflow class method.
ok thanks, I was thinking of having it separate method so that if activity fails and say service 1 is completed then it should restart from service 2 sync and not try to re-sync service 1. Does that also mean each sync service should be in its own activity?