Issue with CRON schedule not working properly in other environments

Community Support
,
1

First of all, Temporal is a great platform and I would like to thank you all for being active and answering all my questions and helping me solve my issues.

My issue has something to do with CRON schedule not working properly on our staging and prod environment. While locally, it works as expected.

I set it to run every 10 minutes with:

String cronSchedule = "*/10 * * * *";

WorkflowOptions options = WorkflowOptions.newBuilder()
        .setTaskQueue(taskQueue)
        .setWorkflowId("SingleWorkflowId")
        .setCronSchedule(cronSchedule)
        .build();
SingleWorkflow workflow = client.newWorkflowStub(SingleWorkflow.class, options);
workflow.runWorkflow();

When I check locally, using the temporal console (http://localhost:8088/), I can see that the Workflow runs every 10 minutes meanwhile it runs every minute on our staging and prod.

Is this an issue or am I missing something on my setup? Thanks in advance

2

Are your staging and prod running the same server version (which version is it)?
Server also supports robfig cron lib and you can set cron schedule to a duration, for example

.setCronSchedule(“@every 10m”)

test that out to see if it makes a difference. I believe this has to do with step values (“/”) in your cron def.

3

Thanks @tihomir , I’ll try that and let you know

4

Hi @tihomir, I tried your suggestion and it worked locally but unfortunately, the problem still exists on our staging environment.

5

Can you show your workflow execution history (on local and on staging env). This would help trying to figure out what could be going on .

6

Here are the workflow execution. I have no idea why the CRON schedule is different between local and staging. I have the cron schedule hardcoded.

WorkflowOptions options = WorkflowOptions.newBuilder()
        .setTaskQueue(taskQueue)
        .setWorkflowId("AutoScalingWorkflow-ID")
        .setCronSchedule("@every 10m")
        .build();

Here are the different execution info from tctl workflow show:

Local:

image
image774×244 33 KB

Staging:

image
image698×168 30.4 KB

7

Can you check if on staging this workflow execution with AutoScalingWorkflow-ID id was already running before you try to start it in your test? Your client call to start it again with same workflow id should fail (you can only have one execution running with same id at a time).

If you are running on server version 1.16.0 + try setting in your workflow options:

.setWorkflowIdReusePolicy(WorkflowIdReusePolicy.WORKFLOW_ID_REUSE_POLICY_TERMINATE_IF_RUNNING)

to see if this will terminate the currently running cron execution then check the cron schedule in WorkflowExecutionStarted event again.

8

Updated and used that setting. The problem was the old workflows was still running so I changed the name of the task queue and updated the workflow id. Looks like the CRON job works as expected now. I’m just wondering if there’s a tctl command to clear the old workflows. I tried tctl admin wf delete but it didn’t remove the old workflows