I’m thinking about an IoT scenario and wondering if Temporal is the right/wrong solution.
There would be a long-lived workflow (e.g. weeks). The workflow would have a few signals associated. Every 10 seconds, the workflow would receive a signal. There would be initially 20k workflows but the system may scale to 500k workflows. Processing a signal would be relatively light (i.e. minimal CPU/memory usage).
I’m concerned about several things. Wondering if someone could help me think through them.
Could I set Temporal to NOT store the signal in the workflow history? It isn’t a case of PII but rather I don’t care about the data and am concerned about history storage requirements. Maybe this is a moot point as signals are persisted to storage?
How do I calculate how many workers my system would need? I’m unsure what would be the bottleneck in the system. I would like to try to figure it out before deploying to production/staging/etc. The associated costs may be a blocker to design acceptance.
Can I crop/limit workflow history for a running workflow? Again, I’m concerned about storage requirements and don’t care about the history of workflows.