Monitor/handler for missed scheduled Workflow

[vutunam]

Hi Temporal Team, I’m trying schedule feature of Temporal (temporal-sdk 1.21.2 - Java)

Currently I don’t see any feature to check for missed schedule workflow, please advise me to handle the missed cases (at least retry to re-run them):

Example 1 - Simulate worker failure

Here I run my schedule worflow every minute (using setCronExpressions(" * * * ") ), let my worker down for few minutes between 17:18 to 17:21, and then start worker up again

→ Missed workflow runs at 17:19, 17:20, 17:21, but there’s nothing to indicate it, nor any action to make up for missed runs

Example 2 - Stress test

Miss hosted at ImgBB

Here I stress test Temporal with 8000 workflows run the same time - every minute (using setCronExpressions(" * * * ") ), very light weight activity and using temporal in memory database

Of course, we cannot handle all 8000 workflows every minute, and the execution times are messed up

→ Missing workflow runs, no options to make up for those missed

[antonio.perez]

Hello @vutunam

I think setting Overlap Policy to SCHEDULE_OVERLAP_POLICY_ALLOW_ALL should solve both cases.

Additionally, you can triger schedules manually or run Backfill

Let me know if it helps,
Antonio

[vutunam]

Thank you @antonio.perez

I have tried the overlap policy and it worked fine.

If a missed workflow run later on using overlap policy, can I get that workflow SCHEDULED TIME inside @WorkflowMethod (I can see that it appended to workflowID (scheduledTransfer_3-2023-10-16T04:14:00Z) but is there a better way to get it) ?