Hello, is there a way to set a retry timeout?
We have an activity that sends a message to an external system. The external API may periodically report a 429 code indicating the time of the next attempt. This time is always different.
I understand that this is an application level error, and the task of Temporal is more likely to handle failures of the platform level, network, etc.
Maybe there is some advice? It would be ideal if the process itself could react to such failures and handle them by sleeping for a while or restarting the activity in accordance with the time that came in the 429 code.
You cannot dynamically update the default retry policy (or a custom set one) for an activity once you request its invocation.
You can use the second or third recommendation for polling in this forum post.
If you rather not implement the polling inside your activity code, would go with the child workflow approach, have it execute the activity which can return the duration when next activity invocation should be done, have the child workflow use workflow.Sleep for that duration and then call continueAsNew.
Hope this helps.