I am considering building a system with Temporal, essentially we are syncing things to Netsuite.
We will have many workflows within the company, a division may be purchase orders vs sales orders, so each team will have their own workflow.
We have two issues we are trying to reason about
One issue lies in the fact that the Netsuite api has limited concurrent connections, I was wondering if there was a way of limiting this, or perhaps I need a second workflow that just handles the sending to Netsuite
A second issue we have would be ordering, we need to process everything for say Sales Order 12345 sequentially, but we need parallism to ensure we are getting everything processed, we had considered hashing 12345 into a number 1 - x with 1 - x workflows for processing this, which would process one thing at a time, that would mean we would have x concurrency and maintain ordering.
One issue lies in the fact that the Netsuite api has limited concurrent connections, I was wondering if there was a way of limiting this, or perhaps I need a second workflow that just handles the sending to Netsuite
Temporal doesn’t have out of the box solution that limits activity parallelism. The specific workaround depends on the maximum rate of the activity executions that call Netsuite. And what is the maximum allowed parallelism?
Temporal does support global task queue rate limit for activity executions. Could you rate limit calls to Netsuite API instead of limiting parallelism?
A second issue we have would be ordering, we need to process everything for say Sales Order 12345 sequentially, but we need parallism to ensure we are getting everything processed, we had considered hashing 12345 into a number 1 - x with 1 - x workflows for processing this, which would process one thing at a time, that would mean we would have x concurrency and maintain ordering.
This will depend on the specific answer to the first question.
we know the current max limit is 40 concurrent connections, this can be increased with more money, which would be good to not have to do.
currently I think we would struggle to answer the question on how many requests are being made, due to the system being sub standard, lack of observability, we do know the limit is hit on occasion.
We had considered a piece of infrastructure that would act as a token provider.
The other line of thought was to create a new workflow that only sends to netsuite, but that would not allow us to guarantee ordering of events, it would also become a single point of failure.
@Charles_Bryant we needed to implement rate limits recently, we did it with a very simple wrapper that uses Redis expiring counters. A rate-limit “error” (i.e. attempt to go beyond counter limits) would cause an activity to be retried by Temporal with whatever retry options you’ve provided.