We build a SaaS product (think of it as a ticketing system like Zendesk) and are at a stage where we’re building a lot of integrations for our customers (think Zendesk → Slack, Zendesk → CRM, Zendesk → our-customer’s-custom-software, etc).
Our product has a REST API interface and is capable of publishing webhooks. We simply build AWS Lambda applications for each of our integrations solutions, basically consuming our product’s webhooks, doing some transformations, then making external API calls and then updating our system again via API. And then when we want to pull updates from a 3rd party system, we either have pollers setup on Lambda or expose API Gateway endpoints that are registered on their systems to send us updates: we consume these, make necessary transformations and then subsequently call our product APIs to make the necessary updates.
Although this is a workflow, we’ve basically built it as functions that do it all in the same execution context (like each interaction I described above). This means between API calls there is no error recovery mechanism, aside from the exception handling in javascript we’re doing. And interms of visibility, we only have at a high granularity level and not at every step of the workflow. And obviously we need to build a lot of ancillary concerns like logging, heartbeats, circuit breakers, etc (which we have not done completely yet).
One solution for us would be to go with something like Mulesoft or Tray.io or n8n.io. But then there are cost considerations and I really like the way you can build workflows as code with Temporal.
So I’m wondering if Temporal would be a good solution for us to build out these integration workflows? Our platform itself is built with Ruby on Rails and we don’t have any control over the 3rd party systems we will be invoking. So Temporal workers would just be orchestrating API calls.
I’m guessing we would need some component that can consume webhooks from our application and then initiate Temporal workflows as I understand there’s no way to expose a HTTP endpoint directly from Temporal server (I’m very new to this, so I might be wrong).
What are the tradeoffs we should be considering here? Is this something other teams are doing? Happy to provide further details if that would help.
(Also, is this the correct forum to ask such questions or should I go on Slack or book office hours?)