Java SDK Threading breakdown

Community Support
, , ,
1

Hello Colleagues,
Could you please help me to understand threading system for java SDK.

Activity settings:

    workers:
      - task-queue: TQ-MSG-BACKEND-PREPARE-ACTIVITY
        name: messageBackendPrepareActivityWorker
        capacity:
          max-concurrent-activity-task-pollers: 10
          max-concurrent-activity-executors: 16

i expected 10 pollers threads and 16 worker threads but result is

image
image1920×1346 400 KB

pollers - as expected
workers - it spawns a lot more

is it some kind of miss configuration or it works as intended?

Workflow settings:

    workers:
      - task-queue: TQ-TXN-PAYMENT-EVENT-PUBLISH-WORKFLOW
        name: txnPaymentEventPublishWorker
        capacity:
          max-concurrent-workflow-task-pollers: 10
          max-concurrent-workflow-task-executors: 32

worker we have huge amount of threads

workflow[...]-...
workflow-method-...

is it cache? what is the difference between workflow and workflow-method? So big numbers of threads looks terrific as we have boxes with only 16 cpu.

2

I don’t know if this is why you see extra threads and might be unrelated, but note that the Temporal Java SDK uses threads in order to allow a workflow to have multiple call stacks at the same time – while only running one of those threads at a time because workflow execution is single threaded.

For example, consider a workflow which logs “before sleep”, then calls Workflow.sleep(), and then logs “after sleep”. At the point where it’s calling sleep, there’s a Java call stack so that after the sleep returns execution will continue with calling log “after sleep”. Meanwhile suppose there’s a signal handler that runs an activity, which runs while the first execution is paused in the sleep. The execution in the signal handler has its own call stack.

Some other languages have features like generators which you can use to get multiple call stacks, but in Java the only way to get multiple calls stacks is by using threads.

So again, I have no idea whether this might the source of the extra threads that you see or not, but in workflow executions at least, the multiple threads don’t consume CPU because only one thread is executing at a time.

3

java thread is not only memory resource, i would doubt that this is free, even memory can cause OOM. Is there any limitation for these threads or can i force sdk to use thread pools instead of creating huge amount of threads?

for workflow i found the way to setup workflowCacheSize >> maxWorkflowThreadCount(like workflowCacheSize=10000 maxWorkflowThreadCount=32), but it is against recommendation. Could anybody explain system behaviour for this case ?