Making DB calls and rate limiting ReST calls in a multi-worker environment

Community Support
, ,
1

Hello Temporal Gurus,

We have the following workflow to be implemented in temporal.
Workflow:
{
make_db_call();
invoke_ReST_API_1(); //rate limited to 10 calls per sec
invoke_ReST_API_2(); //rate limited to 5 calls per sec
send_kafka_message();
}

Given the above, we have the following questions:

  1. Is there a way to ensure workflow for the same input data is identified and not processed?
  2. Is there any limitation on making the DB call #1 in a workflow activity?
  3. Is there any Temporal best practice for implementing distributed rate limiting for #2 and #3?

Thanks for your help.

2
  1. Workflows are unique by WorkflowID. So construct the workflowID from data to satisfy your requirements.
  2. All external API calls should be done from activities. So your workflow should call four different activities for all the external API calls.
  3. You can configure per task queue rate limit. If you need to specify different rate limits for different activities use a different task queue for each such activity.
3

Appreciate your timely response, will try this out.