Long polling inside workflows or activities

So we have an order use-case and an external system is expected to process payment, the customer has X days to settle payment using the external system (the amount varies by types from 2-0 days)
during this time the order status is Pending-Payment …

I was planning to create a workflow that will poll the internal system every hour or 30 min maybe even some exp. backoff config that will identify if the payment was made and if so move the Order into Approved status.

I cannot make the activity poll because then the worker will be busy forever and its illogical to idle the worker for such a long time also I believe setting such a long timeout is not practical

Creating a workflow that will poll in a loop using an activity will possibly create a huge workflow event history and might exceed the limit…
In addition I’m not sure it falls into the deterministic constraints requirement because some executions will have 3 polls until completed and other will have 300 …

Would love to hear any feedback or direction for implementing this.

Thanks for the quick reply …
So I was thinking of utilizing retry policy for that … the thing I didn’t like that the workflow will display failures of activity when the poll was an actual success and there is no simple way to distinguish between an actual failure to poll (due to conn err or config issue) vs a successful poll with a negative answer …

There is another issue @maxim with this approach.
After poll timeout is reached and the payment has not been fulfilled there is some cleanup work to be done (remove the payment request)

to achieve this with retries i will have to have some code knowing how many retries to expect and if its the last one i will issue a call to another workflow to verify that payment request was remove …

all this does not seem elegant …

from the determinism approach, can i invoke an activity in a loop a different amount of time with workflow.sleep(…) between iterations ?

i know the history might be big … but ill make sure to have the poll to be set and validated for a reasonabla amount of retries … no more than 50 i think …

Sorry, I don’t understand your concerns. After poll timeout is reached the activity will complete/timeout and the workflow can execute whatever cleanup logic is needed. What is not elegant about performing the cleanup logic in a finally block?

Ha, sorry it is I who didn’t understood what you meant …
We wrapped the sdk with a generic Simple workflow of a single activity … what you mean is I create a workflow with 2 activities one for polling with retries … and then based on its outcome another activity with cleanup logic.
Thanks . Yest that will work … will have to create a new wrapper then :slight_smile:

In our project we decided not to openly let writing of complex workflows, we create dedicated wrappers for common scenarios… so this will be an implementation of a polling workflow with pre, poll and post activity …

Thanks for the help !

I would advise against such a wrapping. It sounds like a good idea, but it just eliminates 90% of Temporal value without much benefit.

1 Like

I agree that it eliminate most of temporal capabilities …
But this is exactly my goal. Make it hard to do complex things and stick to known patterns …

At least for the beginning of temporal integration into our codebase.

There are 3 scenarios we support …

  1. Simple single activity workflow to be executed
  2. Scheduled Cron like job
  3. And now a polling workflow …

As we dive deeper into implementation we will consider opening up more flexibility …

Main concern is code maintenability

The main goal of Temporal is to eliminate the need for “known patterns,” as all these patterns are extremely leaky abstractions. The code maintainability of a 100% Temporal solution is much higher than that of any EDA-based system.

I’m talking from my experience with many teams that introduced Temporal. The teams that tried to hide Temporal from developers were the most unhappy ones in the long term. The exception is when the end users are not developers; then, a DSL-based solution makes sense.