Update search attributes when activity retry happened?

Community Support
1

Hi, is it possible to update search attributes when a activity do retry?

We wanna to monitor worker and workflow status when start a branch of workflows. Such as,

  • How many workflows on taskA “scheduled”
  • How many workflows on taskA “processing”
  • How many workflows on taskA “completed/failed”

Here are workflow code

1. UpsertSearchAttributes state taskA "scheduled"
2. ExecuteActivity "TaskA" 
    2.1 TaskA activity code will call signal workflow first
3. UpsertSearchAttributes state taskA "completed" or "failed"

To support state “processing” will be a little tricky:
Workflow code start a thread to handle signal background wait for signal coming, and upsert search attributes

But when Activity failed and backoff retry, we wanna to find a method to reset or update search attributes state to scheduled again.

Is there any idea can help?

Thanks.

2

We plan to add such functionality. I don’t think we will update the search attribute on the first retry as it can generate significant update traffic without much value. But updating it after retries exceeded some threshold makes perfect sense.

3

Nice to hear that.

But for long running task (maybe consume a few hour), the update traffic is acceptable.

4

That’s why the threshold should be configurable.

5

I have the same question. Search attributes would be wrong if the task is retried:
1.taskA scheduled
2.taskA processing
3.taskA failed and retry
4.now taskA at processing state
but in fact, task A should be scheduled state since there is no worker polled it.

6

You should not use search attributes for high rate of updates. So the state would be more like “activity didn’t complete within a specified timeout” and you’ll need to look at the specific workflow details to find out its exact state.

7

@maxim Is there an alternative to upsert custom attribute if the activity is retried? I tried using try except but attribute is not being updated. Before you told that you plan to add this functionality, do you have the link to ticket which request to add the funcionality. Thanks in advance!

8

I would be careful about updating the search index on every retry. It can easily overload it.