How to add new PollerOption to workflow?

Community Support
1

Hi, I already created new PollerOption and want to attache this new PollerOption to workflow. How I can do that?

PollerOptions pollerOptions = PollerOptions.newBuilder()
.setMaximumPollRateIntervalMilliseconds(2000) //default 1000
.build();

Best regards Igor

2

Hi, you can set this via WorkerOptions, for example:

Worker worker = factory.newWorker(TASK_QUEUE, WorkerOptions.newBuilder()
  .setMaxConcurrentWorkflowTaskPollers(X)
  .setMaxConcurrentActivityTaskPollers(Y)
  .build());

see here for more info.

1 Like
3

Hi @tihomir
I am getting an error “observation: RESOURCE_EXHAUSTED: namespace concurrent poller limit exceeded” . Would the above configurations work in this case ?
Also, in the Java SDK Client , the above methods “setMaxConcurrentWorkflow/ActivityTaskPollers” don’t seem to be available , has this configuration changed ?

4

RESOURCE_EXHAUSTED: namespace concurrent poller limit exceeded

In this case you could

  • Reduce the number of your pollers / workers polling on this namespace
  • Increase frontend.namespaceCount in dynamic config (default 1200) for this or all namespaces if you wanted
  • Add more frontend pods ( im assuming you have > 1 and are load-balancing them)

For second question, which sdk version are you using? Both should be in WorkerOptions

1 Like
5

Thanks Tihomir, we are currently using Temporal Java SDK 1.16.0 . A few related questions.
Q1. To reduce the number of pollers/workers polling on this namespace - can I use the setMaxWorkflowThreadCount() API ? - This is currently set to the default value of 600 - does this still need to be reduced ?

 /** Java SDK **/
 private static final int DEFAULT_HOST_LOCAL_WORKFLOW_POLL_THREAD_COUNT = 5;
 private static final int DEFAULT_WORKFLOW_CACHE_SIZE = 600;
 private static final int DEFAULT_MAX_WORKFLOW_THREAD_COUNT = 600;

Q2. frontend.namespaceCount - What is this counter used for ? is there a way to calculate the appropriate lower-bound for this w.r.t number of active workflows ? Also, is there a way to update this value using the Temporal SDK ?

Q3. We currently have a single pod running Temporal frontend. Should the number of pods for Temporal frontend be based on the mean CPU utilization of the frontend pod ?

6

Q1

For poller counts you would look at WorkerOptions->setMaxConcurrentWorkflowTaskPollers (default 5) and WorkerOptions->setMaxConcurrentActivityTaskPollers (default 5)
WorkerFactoryOptions->setMaxWorkflowThreadCount is setting for max number of actual threads available to be used for workflow executions across all workers created from a particular factory.

Q2

frontend.namespaceCount can be configured in dynamic config (default 1200). It can be filtered by ns and is typically looked at when you see resource exhausted issues for example:

RESOURCE_EXHAUSTED: namespace count limit exceeded

meaning there are too many concurrent polls for workflow/activity tasks to a single frontend host. So it has to do more with the number of workers, pollers and number of frontend hosts, rather than number of workflow executions (per namespace). No, dynamic config values cannot be updated from SDK.

Q3

Temporal server frontend service is for the large part responsible to route requests to history and matching service, and read the workflow event history for your sdk workers, so it is CPU heavy, yes.

1 Like
7

Thanks a lot Tihomir! This clarifies everything!