Disambiguating concurrent workflows called via aysncio.gather()

Community Support
1

I’m in the early stages of creating workflows & activities using the python SDK. My use case requires invoking multiple concurrent workflows with asyncio.gather(). My activities are written to return an output dataclass with attributes that include a tracking identifier. This works great as long as the activity doesn’t fail for any reason.

However, if an activity fails for any reason (timeout, or any other failure scenario), then the returned output dataclass code path is never reached and I get back a generic workflow failure exception with no identifying information.

I can work around this by wrapping every activity in an exception handler that catches the top level temporal activity failure exception and returning my output dataclass with internal failed status attribute. But this approach that means that the workflow itself is treated as not failed, which is not great from a reporting perspective.

I’d like to get back an output data that uniquely identifies the workflow result regardless of its state. Is there some way to get the best of both worlds?

2

Yes, we do not add extra information on temporalio.client.WorkflowFailureError itself to identify the workflow. If you have many workflow calls and need to know which call raised the exception (regardless of whether its WorkflowFailureError or some other client error), you should have a try/except caller side that wraps with whatever extra information you may need. Just like if it was some other Python call that threw an exception and you wanted some other contextual state about the exception when viewed in aggregate.

There are lots of ways using asyncio.gather or other Python aggregate calls to know which errored. You could set return_exceptions=True and match the index of the exception w/ the index of the sent-in task. As mentioned above you could make a wrapper call client-side to the workflow that traps/wraps the error with whatever contextual state you want. You could use asyncio.wait with these as tasks and handle exceptions as they come with some mapping from task to workflow.

3

Thanks Chad for confirming the options & status.