Scenario: Workflow with Frequent Signals Received

Community Support
1

Hi All,

I’m thinking about an IoT scenario and wondering if Temporal is the right/wrong solution.

There would be a long-lived workflow (e.g. weeks). The workflow would have a few signals associated. Every 10 seconds, the workflow would receive a signal. There would be initially 20k workflows but the system may scale to 500k workflows. Processing a signal would be relatively light (i.e. minimal CPU/memory usage).

I’m concerned about several things. Wondering if someone could help me think through them.

  1. Could I set Temporal to NOT store the signal in the workflow history? It isn’t a case of PII but rather I don’t care about the data and am concerned about history storage requirements. Maybe this is a moot point as signals are persisted to storage?

  2. How do I calculate how many workers my system would need? I’m unsure what would be the bottleneck in the system. I would like to try to figure it out before deploying to production/staging/etc. The associated costs may be a blocker to design acceptance.

  3. Can I crop/limit workflow history for a running workflow? Again, I’m concerned about storage requirements and don’t care about the history of workflows.

Thanks!

Albert

2

Every 10 seconds, the workflow would receive a signal.
I don’t care about the data and am concerned about history storage requirements

What is the reason for sending a signal every 10 seconds without caring about the data in it?

3

Oh, sorry. Typo. I meant “I don’t care about the data retention”.

To add a bit more context… The signal will be lat/long position updates that could potentially “trigger” a geofence event.

4

I see. In this case I wouldn’t signal workflow every 10 seconds. I would have an external component that would listen for events and only signal workflow when the geofence event is triggered.

5

Just wondering, why would Temporal.io be a bad fit for this? I was hoping to use it to centralize geofencing logic as there will be many geofences associated with one device such that each device would have its own geofences (e.g. different home locations, different go/no-go areas, etc). Registering and deregistering these geofences would be nice to do in Temporal.io…

6

The Temporal is good for managing state reliably. Sending a bunch of events that are ignored 99.9% of the time is putting a lot of load on DB without any direct benefits.

You can still use workflows and organize the geofencing logic using Temporal. I just think that moving the filtering logic out of the workflow code makes sense in this case.

7

Thanks for clarifying!