Hello,
I have a long running activity that launches a job on another compute platform and polls for the status until completion. I have a requirement to retry submission of the job n in case of failure. So to have the idempotent activities, I have broken my activity into two: launchActivity followed by watchActivity. Both have their own retry logic.
The problem I am facing is with reset of the workflow that has this activity from the point of failure. If the launchActivity fails, it is easy to get the eventId of the scheduledEvent and launch the job again. But if the watchActivity fails(due to job failure on 3P compute platform), How do I get the eventId of the respective launchActivity?
Why do you ever fail these activities? Keep retrying them forever and reset is not needed.
Failures are non retryable in the current context. For ex: we are launching a compute job that has a logical bug and needs to be fixed. But we all the other job prior to it in the workflow are successful so we want to reset from this specific job.
If the job is modeled as an activity you can deploy the new version of the job and it will go through on the next retry.
With temporal even failures that require new code deployment can be treated as intermittent (retryable).