How to and the best practices for logging to external systems (slack, telegram, sentry, ...)

Community Support
1

Using the Python SDK, I have some workflows and activities, and I want my logs (warning logs only in this case) to also be sent to an external system (slack in this case), so I need to send an http request to slack. For all my workflows and activities, regardless if succeeded or not.

The simplest solution would be to create an activity (send_to_slack) and call it at the end of each workflow I want. However, with this solution I would need to aggregate all the relevant info and send it in one batch in the last activity of the workflow. If the workflow fails before that, it will not work and send nothing.

In my research, I found the example of Python SDK that uses interceptors (with Sentry), but the example doesn’t make it clear that handlers correctly deal with non-deterministic code. I tried but got a lot of errors when requests, even using the imports_passed_through and sandbox_unrestricted methods options to deal with non-determinism.

I also found that there is a sink option/alternative (Is it possible to intercept Workflow exceptions and send them to Sentry? - #2 by jwatkins) but from what I found, it only exists for the Typescript SDK.

I also tried attaching another logging handler (custom logger) to the existing loggers (activity and workflow) but ended up with the previous errors of non-determinism and also metaclass problems in python when handling with the requests library inside the logger emit method.

There is a solution to make HTTP requests in interceptors or loggers (handlers)? Will the sink option eventually be available to Python SDK? Could it be a valid solution for my case?

Or is calling it inside an activity the only possible solution for now?

Here are some parts of errors that I found (even using imports_passed_through and sandbox_unrestricted methods options):

...
File "/app/.venv/lib/python3.11/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
...
File "/app/.venv/lib/python3.11/site-packages/temporalio/worker/workflow_sandbox/_importer.py", line 234, in _import
  mod = importlib.__import__(name, globals, locals, fromlist, level)
...
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

Thanks for the help.

2

Call the send to slack activity from the workflow interceptor. The workflow interceptor is technically part of the workflow code. So it has the same deterministic constraints.