How to detect a cron run timeout and alert?

I followed the example here
The cron runs every minute. I modified the execution and run timeout to this
I let it run for 2 minutes and then killed the worker and here is what I got

How can I detect that the run timeout happened and raise some alert? And what if I want to alert only when 2 consecutive run failed/timed out?

Thank you in advance!

We emit a workflow_timeout metric tagged with namespace. Unfortunately this is not hooked up for cron workflows yet. Issue#397 tracks this improvement. Once it is fixed you should be able to use this metric to track cron workflows timing out.

Thank you very much samar
Is there a doc around how to use metric to setup monitoring and alerts?
Or I should just learn one of the systems listed here?

Lots of users are running Temporal with prometheus as metric backend. Please refer to our helm-charts for documentation on setting it up. If you search on forum you will find a lot of information on this topic. Few examples:

We are using Datadog. Any support on that.
Google search shows nothing and search datadog in the forum got nothing.
I’ll ask a separated question