Hi everyone. I’m brand new to temporal and just trying it out on a toy use case. I have a workflow that is tracking the state of a payment. In the case of a bank payment, this is a long running process that could take days to clear. There is metadata around the payment but importantly the workflow keeps track of the status of the payment which could be “Success” | “Pending” | “Error”.
What would be the best way to find out say, how many payments are currently pending? I could have an Activity to write the current state to a sql database whenever the status changes and then query that database but that feels like it goes against the purpose of keeping state in the Workflow.
Is there some method or architectural pattern of “bulk workflow querying”? Another example of a useful query would be to list the history of all payments made by a given user (assuming user metadata exists in the workflow).
Does this mean that my problem fundamentally doesn’t fit temporal pattern?
The Temporal query feature only works for a single workflow. Temporal supports listing workflows, but this feature’s main goal is to support UI, so its capabilities are pretty limted and it cannot be used for high query rates.
Using an external DB that is updated by an activity for cross-workflow queries is the way to go.
I was afraid of that. It seems like it would be inefficient to have an activity write out every state change. I’d essentially be keeping 2 separate copies of the data that also need to be kept in sync.
Do you have any suggestions for other methods to achieve my goal? Maybe it would make sense to first write to db then signal the workflow?
You’re right. I wouldn’t need to query for every state change but I could query at any time and need the (external) state to be accurate.
In the originally stated scenarios each Payment is its own workflow. At any time an admin could want to see a dashboard of all payments that are pending. Or a user could want to see a list of their payment history.
So I guess the common theme is to find the correct workflows and then do individual queries for each matching workflow. Maybe It would make sense to have a hierarchy of workflows? A user workflow which would keep track of which payment workflows are associated w/ a given user. An admin workflow which keeps track of which payment workflows are currently pending.
I feel like I’m kind of dancing around the root of my question. So far the answers seem to be having a hierarchy or using an activity to write to db but I’m having a hard time visualizing what that looks like specifically.
It seems like for most non-trivial systems, there will need to be some data-spanning queries. Take for example the ecommerce repository. I would imagine there are further requirements for the system where the business can see a list of abandoned carts and a user can see a list of all their past carts/orders. How would you modify the ecommerce repo to meet those 2 requirements?
I’m not familiar with the ecommerce repository, so it is hard to answer your question. I understand that a sample would be nice, but the general idea is I described earlier:
Store the state of an individual workflow in its variables.
Use workflow query functionality to query data about a specific workflow.
Use DB to store records needed for listing workflows and aggregating data across them.
Use activities (consider using local activities) to update these records.