Namespace Rate Limit Exceeded

Hi,

I’m encountering the RESOURCE_EXHAUSTED: namespace rate limit exceeded error during a load test.

I’m currently testing with a 24 TPS load and have set frontend.globalNamespaceRPS to 50. The namespace rate limit is set higher than the actual load, but the rate limit exceeded error still occurs. I have even tried increasing it to 500, yet the error persists.

From the metrics, RPS limit exceeded happens for {operation="PollWorkflowExecutionHistory", resource_exhausted_cause="RpsLimit"}.

Is there any metrics to measure the actual RPS so that I can determine the correct value for the namespace RPS? How could it be that the RPS limit required is way higher then the actual load?

I would appreciate some guidance on this.