Full example of Python sync worker

Community Support
1

Hi there.

There’s a documentation on Temporal Python SDK synchronous vs. asynchronous Activity implementations

However, I couldn’t find a full working example of the worker to run synchronous activities.

Basically, what I want to achieve is to:

  • run multiple workers running synchronous activities in a single-threaded single-process fashion
  • deploy them to k8s
  • scale them via HPA
  • when this setup (baseline) is stable, consider improving performance either via async or by leveraging concurrency (thread/process pool).

Using money transfer as an example, this is a code I’m talking about:

async def main() -> None:
    client: Client = await Client.connect("localhost:7233", namespace="default")
    # Run the worker
    activities = BankingActivities()
    worker: Worker = Worker(
        client,
        task_queue=MONEY_TRANSFER_TASK_QUEUE_NAME,
        workflows=[MoneyTransfer],
        activities=[activities.withdraw, activities.deposit, activities.refund],
    )
    await worker.run()


if __name__ == "__main__":
    asyncio.run(main())
  • I assume I’d need to add activity_executor (Thread Pool or Process Pool) to the worker config.
  • Should it use asyncio library, async functions and await at all?

Thanks in advance

2
3

@maxim so just to clarify, it’s okay to run the __main__ and the worker as async, but the actual activities should be defined as non-async and the activity_executor should be defined. And then these activities will run as normal blocking functions on that executor.
Correct?

4

Yes, I believe this is correct. Have you seen this?

5

@maxim I have seen this. In fact, I referred to this link in my first post. And it was a source of my confusion. The thing is, this documentation page lacks the full example, it has only excerpt with activity definitions, not the worker __main__ method.
And in my limited experience, it’s rather rare to mix async and sync methods in the same runtime, hence I wanted to double check if this is by design that the worker still should be started via async and work as such, but the activities will run in another executor pool in synchronous fashion.

1 Like
6

I think I have found the answer from Temporal with synchronous python services

Basically, your worker will still start in async mode and the workflows will still be running in async manner. However, you can run the sync activities by providing a threadpool when you create the worker. Ex Temporal with synchronous python services

    async with Worker(
        client,
        task_queue=<queue name>,
        workflows=<list of workflows>,
        activities=<list of activities>,
        activity_executor=ThreadPoolExecutor(5),
    ):

Then you can write all your activities without async keyword (aka running them in synchronously).

2 Likes
7

@nakarin29 that was my initial assumption in my first message :slight_smile:

I assume I’d need to add activity_executor (Thread Pool or Process Pool) to the worker config.

I hope this is the proper way, and this is my understanding so far (temporal client runs in async, but sync activities run on a separate pool (thread/process)).

Would be nice to have a complete example, as this is not a frequent and obvious combo (async+sync together).