I’m new to Temporal but find it very interesting. We are looking at using it for the next rollout of our platform scheduled for end of year so I’d like to get an idea of what’s the best way to architect it.
- We have ‘spikes’ in demand sometimes - for example an influx of leads for which we need to perform a set of workflow steps. Would it be recommended to put Kafka as an incoming queue so we can handle spikes?
- Per-tenant (a tenant is a customer - an outside organization or agency) we sometimes need to limit the number of leads flowing through the workflow at a given time. Does this mean I need to create a long-running workflow per tenant and send it leads through signals? Or should we use per-lead workflows and would there be some way to manage them?
- For rate-limited tasks like sending emails we are thinking of making an async activity and pushing them onto an outbound Kafka queue. Is there a way to do this directly in Temporal instead?
The architecture we are thinking off looks like this:
(External Workflow Triggers) -> Inbound Kafka Queue | | V Temporal ---- (listens on Kafka and start workflow for each trigger) | | V Outbound Kafka Queue
Any inputs/suggestions would be appreciated. It would help us decide if Temporal is a good fit and how best to use it going forward.