You cannot dynamically update the default retry policy (or a custom set one) for an activity once you request its invocation.
You can use the second or third recommendation for polling in this forum post.
If you rather not implement the polling inside your activity code, would go with the child workflow approach, have it execute the activity which can return the duration when next activity invocation should be done, have the child workflow use workflow.Sleep for that duration and then call continueAsNew.
Hope this helps.