Graceful Shutdown of temporal workers not marking activity as complete

Hi, the background of this, is this github issue [PLUGIN, TEMPORAL] After the activity is completed and worker is terminated (graceful), the last event is still ActivityTaskScheduled · Issue #648 · spiral/roadrunner · GitHub.

Our use case is like this:

  • We deploy our temporal worker using php-sdk inside a kubernetes cluster using kubernetes deployment.
  • We have a long running activity that we cannot afford to re-run again, which is why we set the retry count to 1
  • Sometimes we have to update the workflow and activity code, for this, we trigger graceful shutdown on the temporal php-sdk
  • Thanks to the spiral team, when graceful shutdown is triggered, the long running activity will run to its completion
  • However, when graceful shutdown is triggered, the activiy result is not sent to the temporal engine, the error code is something like connection is closing
  • I don’t know much about the go-sdk, and whether or not it also suffer from the same issue

I’d love to hear feedback from the temporal team and community.
Is this not possible to achieve ?
I feel that our use case is justified.

Thank you :pray:

@changwuf31 this looks similar to issue reported to the Go SDK - Activity result is not stored (and continued) when stopping worker gracefully · Issue #480 · temporalio/sdk-go · GitHub. I’ll ask the SDK team to take a look. Thanks for reporting this.

1 Like