In a scenario of a particular activity failure in workflow, We need to create a user task and on user decision we have to retry or cancel the workflow. Is there a way in temporal we can achieve this?

I have implemented the custom user task in case of particular activity failure. We are signalling workflow on user decision to retry or cancel. But when we signal workflow with retry we are using continueAsNew option but i see all the succeeded activities are re executing before retrying the failure activity.

Is there a way we can straight way retry the failed activity rather then re executing all the executed activity.