Recently found Temporal, and congrats on a great product! I am interested in resiliency aspects, orchestration / workflows, saga’s, and possibly to provide an actor system. A lot of the docs focus on scalability requirements for large-scale (‘centralized’) platforms.
Use case
I am eyeing Temporal for a decentralized FOSS application that will be part of the Fediverse. There will be many server instances that communicate with other instances and interop with other Fediverse applications using ActivityPub and ActivityStreams (actor-based protocol using JSON message exchange between inboxes and outboxes over HTTP).
A server typically hosts a small community of users (smaller is better) or is even self-hosted by a single person. Each server is a platform, where service modules (plugins) can be downloaded and dynamically added at runtime (was thinking of using Hashicorp go-plugin here). The core service would only provide basic features, with service modules providing (building blocks of) actual applications. Think similar to how Nextcloud platform works.
Prior to finding Temporal I was looking into a domain-driven design (DDD) with service modules corresponding to a Bounded Context or sub-domain and designed according to Ports & Adapters + CQRS/ES. Optional in this design was to have Proto.Actor actor model and have actors in the Application layer to load and cache domain aggregates.
Default install would be for small-scale use, and having a SQLite database. The service modules would run on the same server, so not a true microservices design (though communication uses gRPC). Scaling up and out would be needed in rare cases, and then Postgres is available as db (a large-scale service instance would have 50-100k users).
Considerations
Trying to wrap my head how Temporal might fit, I see a lot of matches.
- CQRS write-side would be covered with workflow + activities packaged in service modules.
- Actor model is possible with Activities in the Application layer that invoke domain aggregates in Domain layer.
- Found a project in the works that deals with SQLite persistence for Temporal data.
Some open questions and things I ponder about:
- With all the focus on scalability and complex microservices architecture Temporal may be overkill.
- I don’t have realtime (e.g. gaming) low-latency requirements. Temporal might be at the core of entire architecture.
- But with that do I have enough control of how data is persisted, or maybe I don’t need to care about that any more.
- How would I go about implementing the read-side of CQRS using Temporal (Kafka suggested elsewhere probably overkill too)?
- I saw that Temporal can be imported / embedded. How involved is that, and will you continue to support that option?
- Would it be complex to replace Elasticsearch with something else (thinking of lieu [1] )?
Curious about any thoughts on this architecture…
[1] lieu community search: https://github.com/cblgh/lieu
(cannot include more links)