Need help with A non-deterministic error has caused the Workflow Task to hang

Community Support
1

I have a wait like following

func wait(ctx workflow.Context) error {
	endTime := workflow.Now(ctx).Add(15*time.Minute)
	for workflow.Now(ctx).Before(endTime) {
		var output string
		if err := workflow.ExecuteActivity(ctx, activity1).Get(ctx, &output); err != nil {
			return fmt.Errorf("%w", err)
		}

		// some condition met then return nil

		// Sleep for a while before checking again
		if err := workflow.Sleep(ctx, 1*time.Minute); err != nil {
			return fmt.Errorf("failed to sleep: %w", err)
		}
	}
	return fmt.Errorf("reached timeout")
}

which is giving non-deterministic error

process event for k8s-cluster-bootstrap-queue [panic]:
go.temporal.io/sdk/internal.panicIllegalState(...)
	/home/runner/.cache/go/pkg/mod/go.temporal.io/sdk@v1.25.1/internal/internal_command_state_machine.go:440
go.temporal.io/sdk/internal.(*commandsHelper).handleActivityTaskScheduled(0xc0011aa300, {0xc0015615cc, 0x2}, 0x11)
	/home/runner/.cache/go/pkg/mod/go.temporal.io/sdk@v1.25.1/internal/internal_command_state_machine.go:998 +0xf8
go.temporal.io/sdk/internal.(*workflowExecutionEventHandlerImpl).ProcessEvent(0xc00a139518, 0xc000bb1c40, 0x3?, 0x0)
	/home/runner/.cache/go/pkg/mod/go.temporal.io/sdk@v1.25.1/internal/internal_event_handlers.go:1092 +0x28a
go.temporal.io/sdk/internal.(*workflowExecutionContextImpl).ProcessWorkflowTask(0xc0006b9170, 0xc0012ce4b0)
	/home/runner/.cache/go/pkg/mod/go.temporal.io/sdk@v1.25.1/internal/internal_task_handlers.go:1074 +0x153d
go.temporal.io/sdk/internal.(*workflowTaskHandlerImpl).ProcessWorkflowTask(0xc000b0e680, 0xc0012ce4b0, 0xc0006b9170, 0xc0106321e0)
	/home/runner/.cache/go/pkg/mod/go.temporal.io/sdk@v1.25.1/internal/internal_task_handlers.go:868 +0x3bf
go.temporal.io/sdk/internal.(*workflowTaskPoller).processWorkflowTask(0xc0004ca240, 0xc0012ce4b0)
	/home/runner/.cache/go/pkg/mod/go.temporal.io/sdk@v1.25.1/internal/internal_task_pollers.go:354 +0x3c3
go.temporal.io/sdk/internal.(*workflowTaskPoller).ProcessTask(0xc0004ca240, {0x32e7920, 0xc0012ce4b0})
	/home/runner/.cache/go/pkg/mod/go.temporal.io/sdk@v1.25.1/internal/internal_task_pollers.go:318 +0x78
go.temporal.io/sdk/internal.(*baseWorker).processTask(0xc000296b00, {0x32e8420, 0xc00117fee0})
	/home/runner/.cache/go/pkg/mod/go.temporal.io/sdk@v1.25.1/internal/internal_worker_base.go:505 +0x153
go.temporal.io/sdk/internal.(*baseWorker).processTaskAsync.func1()
	/home/runner/.cache/go/pkg/mod/go.temporal.io/sdk@v1.25.1/internal/internal_worker_base.go:356 +0x45
created by go.temporal.io/sdk/internal.(*baseWorker).processTaskAsync in goroutine 28
	/home/runner/.cache/go/pkg/mod/go.temporal.io/sdk@v1.25.1/internal/internal_worker_base.go:352 +0xa5

I does not use any Intrinsic non-deterministic logic, then what might be going wrong?
Also the workflow is hanged for infinite time

2

The code looks deterministic. Are you sure that a code change does not cause the error?

BTW you can poll more efficiently without looping inside the workflow. What is the best practice for a polling activity? - #2 by maxim