We’re running into an issue with our Temporal setup where we see the following error counter increase to about 300 errors per second.
service_errors_resource_exhausted{resource_exhausted_cause="ConcurrentLimit", service_name="frontend"}
**
**
I am trying to understand what is causing this exactly and how I can configure an higher upper limit (if possible) in the dynamic config.
By reading the forum I was under the impression that I needed to increase frontend.namespaceRPS or frontend.rps but increasing them N fold does not have any impact so I am assuming I am turning the wrong knob here.
I tried finding ConcurrentLimit in the Temporal github repository but wasnt able to find it on the mainline so either I am running an older version (albeit we started our setup using recent public docker images & the helm charts) or I am overseeing something altogether.
Would love to get some help in resolving our problem!
@antonio.perez @tihomir @Donald_Forbes maybe you’d be able to share your wisdom with us?
Regards, Frank