# Which metric should it track to quickly identify bad deployment

**URL:** <https://community.temporal.io/t/which-metric-should-it-track-to-quickly-identify-bad-deployment/1843>\
**Category:** Community Support\
**Created:** [March 30, 2021, 6:15pm UTC](https://community.temporal.io/t/which-metric-should-it-track-to-quickly-identify-bad-deployment/1843 "2021-03-30T18:15:44Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![madhu](https://avatars.discourse-cdn.com/v4/letter/m/a8b319/32.png) [@madhu](https://community.temporal.io/u/madhu)\
**Post date:** [March 30, 2021, 6:15pm UTC](https://community.temporal.io/t/which-metric-should-it-track-to-quickly-identify-bad-deployment/1843/1 "2021-03-30T18:15:44Z")

</div>

Hi ,

Is there a metric whic i can track to identify bad deployments breaking some long running workflow.

.e.g when i introduce a change which is not backward compatible, i see that the workflow cannot make progress as the determinstiic nature is broken.

and i see error like

> i.t.internal.worker.PollerOptions : uncaught exception
> 
> java.lang.RuntimeException: Failure processing workflow task. WorkflowId=mybroken-workflow-1, RunId=979ceb3a-aae7-445a-920b-04905ed1804a, Attempt=28904  
> at io.temporal.internal.worker.WorkflowWorker$TaskHandlerImpl.wrapFailure(WorkflowWorker.java:349) ~[temporal-sdk-1.0.6.jar!/:na]  
> at io.temporal.internal.worker.WorkflowWorker$TaskHandlerImpl.wrapFailure(WorkflowWorker.java:279) ~[temporal-sdk-1.0.6.jar!/:na]  
> at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:79) ~[temporal-sdk-1.0.6.jar!/:na]  
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0\_261]  
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0\_261]  
> at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0\_261]  
> Caused by: io.temporal.internal.replay.InternalWorkflowTaskException: Failure handling event 91 of ‘EVENT\_TYPE\_ACTIVITY\_TASK\_SCHEDULED’ type. IsReplaying=true, PreviousStartedEventId=89, workflowTaskStartedEventId=802, Currently Processing StartedEventId=89  
> at io.temporal.internal.statemachines.WorkflowStateMachines.handleEvent(WorkflowStateMachines.java:193) ~[temporal-sdk-1.0.6.jar!/:na]  
> at io.temporal.internal.replay.ReplayWorkflowRunTaskHandler.handleEvent(ReplayWorkflowRunTaskHandler.java:140) ~[temporal-sdk-1.0.6.jar!/:na]  
> at io.temporal.internal.replay.ReplayWorkflowRunTaskHandler.handleWorkflowTaskImpl(ReplayWorkflowRunTaskHandler.java:180) ~[temporal-sdk-1.0.6.jar!/:na]  
> at io.temporal.internal.replay.ReplayWorkflowRunTaskHandler.handleWorkflowTask(ReplayWorkflowRunTaskHandler.java:150) ~[temporal-sdk-1.0.6.jar!/:na]  
> at io.temporal.internal.replay.ReplayWorkflowTaskHandler.handleWorkflowTaskWithEmbeddedQuery(ReplayWorkflowTaskHandler.java:201) ~[temporal-sdk-1.0.6.jar!/:na]  
> at io.temporal.internal.replay.ReplayWorkflowTaskHandler.handleWorkflowTask(ReplayWorkflowTaskHandler.java:114) ~[temporal-sdk-1.0.6.jar!/:na]  
> at io.temporal.internal.worker.WorkflowWorker$TaskHandlerImpl.handle(WorkflowWorker.java:319) ~[temporal-sdk-1.0.6.jar!/:na]  
> at io.temporal.internal.worker.WorkflowWorker$TaskHandlerImpl.handle(WorkflowWorker.java:279) ~[temporal-sdk-1.0.6.jar!/:na]  
> at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:73) ~[temporal-sdk-1.0.6.jar!/:na]  
> … 3 common frames omitted  
> Caused by: java.lang.IllegalStateException: COMMAND\_TYPE\_SCHEDULE\_ACTIVITY\_TASK doesn’t match EVENT\_TYPE\_ACTIVITY\_TASK\_SCHEDULED with EventId=91  
> at io.temporal.internal.statemachines.WorkflowStateMachines.assertMatch(WorkflowStateMachines.java:784) ~[temporal-sdk-1.0.6.jar!/:na]  
> at io.temporal.internal.statemachines.WorkflowStateMachines.validateCommand(WorkflowStateMachines.java:737) ~[temporal-sdk-1.0.6.jar!/:na]  
> at io.temporal.internal.statemachines.WorkflowStateMachines.handleCommandEvent(WorkflowStateMachines.java:272) ~[temporal-sdk-1.0.6.jar!/:na]  
> at io.temporal.internal.statemachines.WorkflowStateMachines.handleEventImpl(WorkflowStateMachines.java:199) ~[temporal-sdk-1.0.6.jar!/:na]  
> at io.temporal.internal.statemachines.WorkflowStateMachines.handleEvent(WorkflowStateMachines.java:178) ~[temporal-sdk-1.0.6.jar!/:na]  
> … 11 common frames omitted

Is there a metric which can help me troubeshoot this , i tried temporal\_activity\_execution\_failed but ti think that count does not increase as the activity itself is not invoked.

should i set an alarm on temporal\_request\_failure, with tags namespace=“temporal\_system”,operation=“SignalWithStartWorkflowExecution” to identify a bad deployment?

---

<div class="post-metadata">

**Author:** ![Wenquan\_Xing](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/wenquan_xing/32/362_2.png) [@Wenquan\_Xing](https://community.temporal.io/u/Wenquan_Xing)\
**Post date:** [March 30, 2021, 6:21pm UTC](https://community.temporal.io/t/which-metric-should-it-track-to-quickly-identify-bad-deployment/1843/2 "2021-03-30T18:21:57Z")

</div>

try `service_errors_nondeterministic`