Error when add search attributes

Community Support
1

Our cluster deployed using helm chart https://github.com/temporalio/helm-charts
When try to add a search attribute we got this error

bash-5.0# tctl --namespace default adm cl asa --search_attr_key OrderId2 --search_attr_type keyword
Are you trying to add key [OrderId2] with Type [Keyword]? Y/N y
Error: Add search attribute failed.
Error Details: rpc error: code = Internal desc = Failed to update dynamic config, err: failed to write config file, err: open /etc/temporal/dynamic_config/dynamic_config.yaml: read-only file system.

But after that we list it we do see OrderId2 got added
bash-5.0# tctl --namespace default cl get-search-attr
±----------------------±-----------+
| KEY | VALUE TYPE |
±----------------------±-----------+
| BinaryChecksums | Keyword |
| CloseTime | Int |
| CustomBoolField | Bool |
| CustomDatetimeField | Datetime |
| CustomDoubleField | Double |
| CustomIntField | Int |
| CustomKeywordField | Keyword |
| CustomNamespace | Keyword |
| CustomStringField | String |
| Encoding | Keyword |
| ExecutionStatus | Int |
| ExecutionTime | Int |
| HistoryLength | Int |
| KafkaKey | Keyword |
| NamespaceId | Keyword |
| Operator | Keyword |
| OrderId | Keyword |
| OrderId2 | Keyword |
| RunId | Keyword |
| StartTime | Int |
| TaskQueue | Keyword |
| TemporalChangeVersion | Keyword |
| WorkflowId | Keyword |
| WorkflowType | Keyword |
| orderId | Keyword |
| status | Keyword |
±----------------------±-----------+

But it does not work when we try to use it:
Caused by: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: BadSearchAttributes: OrderId2 is not valid search attribute key

This post say something about schema/elasticsearch/visibility/index_template.json

How do I do it when I’m using helm-chart?

2

Hi Gordon.

First of all, custom search attributes are in “beta” stage. I have plans to rewrite underlying logic early next year.
In your particular case, obvious reason is the dynamic config is read only and can’t be changed. Admin handler change it’s in-memory value and it becomes available for GetSearchAttributes API. This is why you can see it with tctl but it still doesn’t work because history service is not aware of it.

For now, I would recommend you to manually add search attributes to dynamic config and to the schema and redeploy it.

3

Here which walks through the process of adding your own custom searchable attribute.

4

Hi @samar, I saw that post but I cannot figure out how to follow it. We are using helm chart

5

Hi @alex

Thank you very much for those 2 links. For the first one, I think I should put it in https://github.com/temporalio/helm-charts/blob/master/values/values.dynamic_config.yaml

However, I cannot figure out what to do with the 2nd link. What should I do when I’m using helm chart?

Thank you very much!

6

I found this https://github.com/temporalio/temporal/issues/505
After 15 minutes of reading I’m more confused
Do I have to clone the repo, modify 2 files, build my own temporal docker image, publish it and use that in my helm chart just to add a search attribute?

7

I see the dynamic config was only supported recently https://github.com/temporalio/helm-charts/commit/d1191c4d0b867d3de918c3e8e863d9b3918f325e

Is this just a helm chart issue to also have index_template.json mounted just like the dynamic config?

Or should we directly call ES api to manage its index separately?

Very lost. Please help

8

2nd link is the ES schema template. Unfortunately, I don’t know much about helm (even ours). I believe when this step is executed it uses existing template which supposed to be uploaded on ES by then. I don’t know how to upload it with helm. During development process I use install-schema-es target which uploads template first and then creates ES index. I hope there is a better way rather then rebuild docker image yourself but this will work for sure.
As you can see, custom search attributes are not very convenient to use. This is why it is in my short TODO list.