Millions of sleeping workflows

Community Support
#1

I have a use-case where I want to execute a workflow a year after a Kafka event occurs.

Option #1
I can have a worker/cron consume the Kafka topic and persist the events to a table with a creation date timestamp indexes. I can then have my workflow call an activity that will query this table for all events with a timestamp at least a year in the past and perform n the workflow activities.

Option #2
I can have my workflow consume the Kafka topic and sleep for a year before performing the n workflow activities I want to perform.

public void yearLaterWorkflow() {
  val work = getMessageFromKafkaActivity()
  Workflow.sleep(Duration.ofDays(365))
  doSomethingElseActivity()
}

Questions:

  • I expect to have millions of sleeping workflows at any given time. Is this feasible?
  • If need be, can I reach into the underlying Cassandra database and update the sleep time of a workflow retroactively?
#2
  • I expect to have millions of sleeping workflows at any given time. Is this feasible?

Yes, millions of sleeping workflows are not that much.

If need be, can I reach into the underlying Cassandra database and update the sleep time of a workflow retroactively?

It is not possible and not really needed. You can code your workflow to listen to a signal and update the timer. Here are Java and Go samples of an updatable timer.

1 Like
#3

Thank you @maxim :+1: