Handling user feedback, retries, and timeouts

Originally on Slack.

I have a couple of points I want to verify, and questions. They are mostly in the context of the Go client.

  1. On the topic of human interaction, let’s say I need to capture a user’s details (e.g. through a signal), act on it (e.g. through an activity), and give a response to the user. Would the simplest option here be a signal, and query? And is it generally acceptable to query for structs rather than simple strings?

  2. If I have a somewhat long-running activity that fails midway through because the VM running the worker shuts down, I’m guessing the activity will eventually fail assuming it has a timeout? And I can handle this case by setting a retry policy?

  3. What’s the best way to handle a workflow timeout? For example, I want to carry out some compensation action if a workflow times out

1 Like
  1. The simplest is signal and then query. Temporal guarantees read after write consistency for this interaction. Yes, structs are supported as query results. This answer gives more options for implementing synchronous workflow updates.
  2. Correct, the simplest way to retry it is by setting a retry policy.
  3. Don’t rely on workflow timeout for any business logic as they are essentially “kill -9” on timer. Use timers inside the workflow for any business-related timeouts. In your case set a timer that performs cleanup when fired.

I’m giving this a try now.

Assuming I want to have a workflow-wide timeout, with a clean up activity. Would the idea be to use a selector, and a timer like so:

selector := workflow.NewSelector(ctx)
timerFuture := workflow.NewTimer(ctx, workflowWideTimeout)
selector.AddFuture(timerFuture, func(f workflow.Future) {
    if ShouldCleanup {
        _ = workflow.ExecuteActivity(ctx, CleanupActivity).Get(ctx, nil)
    }
})

And for every activity:

f := workflow.ExecuteActivity(ctx, DoSomethingActivity)
selector.AddFuture(f, func(f workflow.Future) {
    _ = f.Get(ctx, &doSomethingResult)
})

Followed by a:

selector.Select(ctx)

So that either the timeout kicks in or the activity completes.

If so, I suppose my next question is, what about error handling within the future? How does that get propagated back to the workflow? Would that be a matter of mutating some err, and checking for it?

I think your approach is unnecessary complicated. I would start a separate goroutine to execute the main workflow logic.

func MyWorkflow(ctx workflow.Context) {
    cancellableCtx, cancelHandler := workflow.WithCancel(ctx)
    workflow.Go(cancellableCtx, func(ctx workflow.Context) {
         MyWorkflowImp(ctx)
    })
     workflow.Sleep(ctx, CleanupInterval)
     cancelHandler()
     workflow.ExecuteActivity(ctx, CleanupActivity).Get(ctx, nil)
}
1 Like

I see! That’s very helpful.

I’m guessing the workflow will continue to run even after the Go routine finishes? And I can make it exit if the Go routine finishes by using a timer, and a channel:

selector := workflow.NewSelector(ctx)
selector.AddFuture(workflow.NewTimer(ctx, CleanupTimeout), func(f workflow.Future) {})

cancellableCtx, cancelHandler := workflow.WithCancel(ctx)
doneCh := workflow.NewChannel(ctx)
selector.AddReceive(doneCh, func(c workflow.ReceiveChannel, m bool) {})

workflow.Go(cancellableCtx, func(ctx workflow.Context) {
    MyWorkflowImp(ctx)
    doneCh.Send(ctx, "done")
})

selector.Select(ctx)

cancelHandler()

workflow.ExecuteActivity(ctx, CleanupActivity).Get(ctx, nil)

This would work. Instead of doneCh you could also use a Future, created through workflow.NewFuture.