Its good practice implement Temporal Workflows to push events to Websockets?

Community Support
1

I have an frontend where is connected to an exclusive service that is an websocket server. This server just listen for redis pub/sub events and publish to your clients via websocket events.

Scenario

image
image1621×897 101 KB

  1. A realtime application sends a signal (event) to an long running workflow (hundred of signals per second).
  2. A workflow child is created for each signal according to signal type received
  3. Executing this child workflow, the temporal worker access redis, get a specific key, make some calcs, and update redis key value (this step is necessary because each pod has its own metric, and the system metric is the result of sum, subtraction, score, etc depending of metric. Ex: pod 1 has 1 call, pod 2 has 2 calls, so the system calls are 3.)
  4. Temporal worker publish to Redis pub/sub the new value.
  5. Websocket server listen the Redis pub/sub and publish to your clients.
  6. Frontend receive the websocket event and update the state of item

Problems

  1. We’ve experienced delays (30s for example) in events depending on Temporal load, and worker availability (sometimes the schedule-to-start time is a little bit more longer than required)
  2. When some calc is not made correctly, the screen “freeze” because worker is trying to execute activity again
2

Temporal is not designed to support this signal rate hitting a single Temporal workflow instance. Can you create workflow per request directly?

3

Yes. I can create using startWithSignal if necessary. Is this a solution?

4

The core limitation of Temporal design is that a single workflow instance has limited throughput. At the same time Temporal scales out practically without limit with the number of parallel workflow executions.

SigalWithStart assumes that you are planning to send multiple signals to the same workflowID. What is the maximum rate per workflowID?

5

Ok. If necessary, I can open an completely new workflow per event . Today the rate per workflowID is 2 per second (QA environment), but it will increase in the future to be something around 500 events per second (production environment).

6

Then, you don’t need signalWithStart as you start a workflow per event. So use StartWorkflow and use a different WorkflowID each time.