Very low throughput with multi docker Temporal Setup

We are trying to deploy temporal and following the below mentioned approach and seeing really low throughput & bunch of errors. We will really appreciate any sort of help here.

  1. We have two VMs each one running docker-compose based Temporal Setup.
  2. We have a PG setup and both of the above mentioned Temporal VMs are connecting to this PG instance.
  3. We have a load balancer and these VM are behind it.

Problem: When I have just machine the throughput is above 100 RPS. When I add two machines, it goes down to 5 RPS.

What are we missing here?