How do you get cleanup logic in between every Activity retry (the Activity failed and is retrying).
I could this by surrounding my Activity in a while loop with try catch in it, but that seems redundant to do with Temporal. Is there a better way?
You can make it part of the activity. You can use the Attempt field in the activity to determine if the cleanup is needed. Retrying from the workflow is the way to go if you need another activity to perform cleanup.