Catching NonDeterministicException

Community Support
,
1

I’ve been bitten a couple of times by making changes to a workflow that caused a NonDeterministicException. I have a use case where when this is thrown, I just want the workflow to fail, so that I can trigger a new workflow with the same inputs on the new version. Basically the desired behavior is when NonDeterministicException just give up on the workflow.

Is there a kosher way of doing this? For example, can I catch NonDeterministicException and just mark it as a NonRetryableException?

2

NonDeterministicException cannot be caught inside your workflow code, but
you could set it to fail workflow execution via WorkflowImplementationOptions, for example:

WorkflowImplementationOptions options =
                WorkflowImplementationOptions.newBuilder()
                        .setFailWorkflowExceptionTypes(NonDeterministicException.class)
                        .build();
worker.registerWorkflowImplementationTypes(options, MyWorkflowImpl.class);

This should fail your workflow in case of NonDeterministicException.
Imo still the default behavior of blocking workflow task allowing you to fix the issue (using versioning for example) might be a better approach than failing the workflow execution.

2 Likes
3

What would be the Golang equivalent for this? I could not find worker.RegisterWorkflowImplementationTypes or anything close.

And is there a way to restart the workflow if the NonDeterministic error happens?

4

Happy to add it if it is not there for go SDK!

5

I think for Go you’d need to set panic policy in WorkerOptions currently, something like

workerOptions := worker.Options{
	// ...
	WorkflowPanicPolicy: worker.FailWorkflow,
}
w := worker.New(ctx, "myworkflow", workerOptions)
1 Like
6

Hi @tihomir ,

A WorkflowPanicPolicy that restarts workflows on nondeterministic error might be a good addition. Do you think that is something I could create a github issue for?

My use-case is this:
All workflows that we write at my company are restartable. i.e, we maintain our own checkpoints so that when a workflow is restarted, it continues from where it left off. We tend not to use workflow versioning since that makes the workflow code hard to read. Hence, an option to automatically restart workflows (a fixed number of times) on nondeterministic error would be super helpful.

1 Like
7

@mvp We have a workaround without any changes to the temporal.
With WorkflowPanicPolicy set to worker.FailWorkflow
And RetryPolicy set to some value when starting a workflow allows us to restart the workflow when workflow encounters non-deterministic panics.

But there are a few caveats:

  • Any panic will result in workflow restarting
  • Any real errors returning from the workflow to terminate the workflow must be wrapped with temporal.NewNonRetyableError
  • Workflow retry does not seem to replay this signals (which we want for our use case)
8

@tihomir is there a way to achieve signal replay when the workflow retries?

9

This is already supported by specifying retry options for the workflow on start.

10

File an issue to add “reset” on the panic option.