Python-sdk parallel activities

I was wondering with the python-sdk if it’s possible to execute multiple activities in parallel. I am looking for a similar capability shown in the go-sdk example - https://github.com/temporalio/samples-go/blob/master/parallel/parallel_workflow.go.

I have tried prototyping using https://github.com/firdaus/cadence-python with Cadence and hoping when temporal python-sdk is already, I can make the switch. So far I am not able to run the activities in parallel. I have tried wrapping activities in asyncio tasks and using asyncio.gather to wait, but the activities don’t execute.

I have a use case for downloading multiple large VM images. I am hoping to have multiple download activities running in parallel to finish faster. Go and Java sdk clearly supports that kind of use case. It’d be nice to accomplish the same with python sdk.

Not yet supported in cadence-python. See “Parallel activity execution” in the roadmap.

Using asyncio primitives won’t work because the activities need to be started in parallel using Temporal’s scheduling features.

Current focus is on porting existing cadence-python features to temporal-python. Once that is done this feature is quite low hanging fruit and lots of people have been asking for it, so i’ll definitely prioritize this.

2 Likes

Thanks for the prompt response. Good point on the roadmap, I missed it somehow.
Yeah, it makes sense to port the existing features first. I understand that you’re working hard on that and your efforts are definitely much appreciated.

2 Likes