Impl details of python worker

Community Support
, ,
1

Hi,
Been reading about temporal, had a few questions regarding worker in temporal python sdk

  • what are the default executors if I don’t pass workflow and activity executors?
  • If I don’t pass an executor and I run a single worker, are workflow and activity executions done in the same python process/worker?
  • If a worker picks workflow task does the same worker execute activity tasks too?
  • Is there a way to monitor active workers count per namespace?
2

For activities, from the __init__ docs:

Concurrent executor to use for non-async activities. This is required if any activities are non-async. If this is a concurrent.futures.ProcessPoolExecutor, all non-async activities must be picklable. Note, a broken executor failure from this executor will cause the worker to fail and shutdown.

For workflows, from the __init__ docs:

Thread pool executor for workflow tasks. If this is not present, a new concurrent.futures.ThreadPoolExecutor will be created with max_workers set to max(os.cpu_count(), 4). The default one will be properly shutdown, but if one is provided, the caller is responsible for shutting it down after the worker is shut down.

Workflow and activity executions are always done in the same process/worker. Technically you can pass a process executor for activities to have activities run in a separate process, but the main task and heartbeating and communication with server and such are the same worker. For workflows, it usually makes no sense to customize the workflow task executor, but it won’t work as a process pool executor nor would it make sense to do so (workflow tasks are CPU-bound operations that run in milliseconds)

Workflows and activities are unrelated. A worker can execute workflow and activity tasks if it is configured with workflows and activities, or only one of those types if it is configured with only one of those types.

Not reliably server side because the server can only know what’s polling and not all workers are polling at the same time (but there are some stale ways to get recently polling workers). Users are usually able to know how many workers they are running.