Question regarding workflow execution count

Community Support
1

On our current temporal setup, we have:

  • a worker that runs in a managed instance group in GCP
  • the worker is started as a debian service
  • each instance in the instance group have a startup script to start a workflow that is setup to run as a cron job every minute

So what happens is if for example we have 5 instances, 5 instances of the worker runs. My question is if it is possible to limit the number of workflows across all worker instances. For example, limiting to only have a single workflow across all instances. I tried using different limits but nothing worked for what I wanted to do. Please let me know how I can fix this issue. Thanks

        Worker worker = factory.newWorker(taskQueueId,
                WorkerOptions.newBuilder()
                        .setMaxConcurrentWorkflowTaskExecutionSize(1)
                        .setMaxConcurrentActivityExecutionSize(1)
                        .setMaxTaskQueueActivitiesPerSecond(1)
                        .setMaxConcurrentWorkflowTaskPollers(1)
                        .build());
2

There is no way to limit the number of parallel workflows. What is the use case?

3

I see. Thanks for the answer. As for the use case, we currently have the workers running in a managed instance group. So the virtual machine instances are duplicates of each other. I think the better solution is to have the worker running on its own vm instance but we just want something simple for version 1.

4

I don’t understand your problem. What is the problem of running multiple workers which are duplicates of each other?

5

Oh sorry, I forgot to mention. I only want 1 instance of a cron workflow for 1 of our workers.

My vm instance right now runs two workers on separate debian services. They are started when the VM is provisioned. Also the workflow is created when the worker is started for each vm.

  • WorkerA - runs core service logic
  • WorkerB - runs a cron workflow

For worker b, we only want 1 instance of the Cron workflow. But right now what’s happening is that we have multiple workflows created during startup.

Also, I realized while typing this that we can also just hold off on creating the workflow and create a single workflow after the workers are running.

6

You can run as many worker processes of the same type as needed for scalability and availability. Uniqueness is achieved through the uniqueness of workflow executions by their ID. Temporal provides a hard guarantee that only one workflow with a given id can be open at the same time. So the solution to your problem is to start the CRON workflow with some fixed ID. Any attempts to start the workflow with the same ID again will fail with WorkflowExecutionAlreadyStarted exception.