I have been using temporal locally on my machine with postgres, elastisearch, auto-setup and ui containers, and ofc another one for running the business logic and contains activity code. I was looking to deploy this on AWS fargate now. I started with the same setup in a VPC except that instead of postgres running in a container I use an RDS. I put an ALB (application load balancer) in front of the temporal ui service so I can access it using custom domain. Once security groups and routing tables are set up all this runs fine.
However I now read that in production we should use temporal-server instead of auto-setup image. However that seems a lot more complicated suddenly. So I was wondering what are the drrawbacks if I continue using auto-setup image instead which handles a lot of boilerplate for me (like schema registration, default namespace creation, elasticsearch visibility configuration etc IIRC).
We are unlikely to run 1000s of workflows in parallel. I think to start with we’ll probably have 10 or 20 in parallel. So if I just create more instances/fargate-tasks running auto-setup image all pointing to the same RDS and elastic search will this not work as expected?
Either:
- The temporal UI can load balance and when it gets multiple IPs for the server (auto-setup) tasks, it can load balance and communicate with one of them and the various server tasks use RDS to communicate with each other etc. OR
- I can put a Load balancer in front of these auto-setup tasks so the temporal ui talks to the load balancer which sends to various server tasks and server tasks know and talk to each other via RDS (my guess).
Will this work? Are there any downsides to scaling the auto-setup tasks horizontally? I want to keep the infrastructure cognitive overhead minimum and not care about history service, visibility service and all the other breakdown of temporal “server” if possible.