Most of my platform has the classic Temporal use case, business logic revolving a lot of non deterministic io calls. Perfect for workflow replay and state.
My problem is with polling. I need to have a very long running workflow, potentially forever lived, that polls a database/queue and then deploys resources for every polled dao.
so basically:
poll from database using an activity
for each polled dao, use an activity/child workflow to deploy the resource.
await a few seconds
My problem is I only need to keep state for the current poll, so if I crash mid poll, I can resume my flow and not deploy the same resource twice, for example. It is completely redundant to save state of previous polls because they have no added value for replay/fault tolerance.
I could continueAsNew after every poll but that doesn’t feel right. The UI will be very bloated from all the continuations.
What is the best way to handle this scenario? Or is there just little value in Temporal for my use case
I still don’t think this issue is resolved. My case would be the last one you’ve mentioned, and I can’t see how a child workflow solves this matter. The main polling loop that calls this workflow would still save redundant state from previous polls for this.
I also must ask, what is the value I have from having the polling loop itself inside an activity? are we not losing the value of temporal by using it like this? I would love to hear about this more
That’s the thing, I don’t have any. I want to save workflow state only for the current poll. In my case, it would be the resulting daos and the deploying activity.