# Custom search attribute doesn't work with temporal server start-dev

**URL:** <https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385>\
**Category:** Community Support\
**Tags:** general-impl\
**Created:** [May 29, 2023, 8:16pm UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385 "2023-05-29T20:16:46Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![karney](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/karney/32/3032_2.png) [@karney](https://community.temporal.io/u/karney)\
**Post date:** [May 29, 2023, 8:16pm UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385/1 "2023-05-29T20:16:46Z")

</div>

Started using `temporal server start-dev` for local development and was able to successfully add search attributes. The search attributes show up when I run `temporal operator search-attribute list` but when the workflow attempts to call `Workflow.upsertSearchAttributes` it gets an error:

io.grpc.StatusRuntimeException: INVALID\_ARGUMENT: BadSearchAttributes: Namespace my-namespace has no mapping defined for search attribute Type

(my-namespace is the name of the namespace and the name of the attribute is called “Type”).

I also noticed that in the Temporal UI when I look at the Search Attributes list in the namespace, instead of the names I gave them, they just show up as “Keyword01”, “Keyword02”, etc.

 ![image](https://us1.discourse-cdn.com/flex016/uploads/temporal/original/2X/1/144053c82be5e7f0a225c9eb3f4cbdbc02fd5493.png)

I’ve tried stopping and restarting the server to no-avail.

---

<div class="post-metadata">

**Author:** ![karney](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/karney/32/3032_2.png) [@karney](https://community.temporal.io/u/karney)\
**Post date:** [May 29, 2023, 8:33pm UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385/2 "2023-05-29T20:33:02Z")

</div>

Seems like using `tctl admin cluster asa ` to register the search attributes works, but using the `temporal operation` command doesn’t. The incorrect search attribute names in the UI persist regardless.

---

<div class="post-metadata">

**Author:** ![taonic](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/taonic/32/5556_2.png) [@taonic](https://community.temporal.io/u/taonic)\
**Post date:** [May 30, 2023, 7:47am UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385/3 "2023-05-30T07:47:46Z")

</div>

Hi Karney,

I can’t seem to reproduce this issue with Temporal CLI 0.9.0.

To eliminate a possibility, can you confirm you are adding the search attribute to the correct namespace `my-namespace` and you can see it from `temporal operator search-attribute list --namespace my-namespace`?

---

<div class="post-metadata">

**Author:** ![karney](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/karney/32/3032_2.png) [@karney](https://community.temporal.io/u/karney)\
**Post date:** [May 30, 2023, 11:37am UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385/4 "2023-05-30T11:37:19Z")

</div>

Earlier, I relied on exporting the environment variable TEMPORAL\_NAMESPACE to set the namespace.

As a test, I removed the attribute using `tctl`, then re-added it using `temporal` specifying the namespace option explicitly and it now works. Perhaps there’s an issue with the env variable not getting picked up, I haven’t tested that.

The output from `search-attribute list` looks the same as previously, so it’s not possible to see any difference there.

The Temporal UI never the less doesn’t seem to display the search attributes for the namespace correctly like the CLI does.

I am on the same version of temporal as you

```auto
temporal version 0.9.0 (server 1.20.2) (ui 2.15.0)

```

---

<div class="post-metadata">

**Author:** ![taonic](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/taonic/32/5556_2.png) [@taonic](https://community.temporal.io/u/taonic)\
**Post date:** [May 30, 2023, 12:01pm UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385/5 "2023-05-30T12:01:40Z")

</div>

The namespace env var has changed to TEMPORAL\_CLI\_NAMESPACE in Temporal CLI [Temporal CLI command reference | Temporal Documentation](https://docs.temporal.io/cli#environmental-variables)

I can see the same UI issue as you reported. If no one else follow up on this thread I can take a look at the code tomorrow morning. It looks like a bug at this point.

---

<div class="post-metadata">

**Author:** ![karney](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/karney/32/3032_2.png) [@karney](https://community.temporal.io/u/karney)\
**Post date:** [May 30, 2023, 8:33pm UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385/6 "2023-05-30T20:33:02Z")

</div>

Might want to also update the output for `temporal env list --help` which currently mentions `$TEMPORAL_NAMESPACE`

 ![image](https://us1.discourse-cdn.com/flex016/uploads/temporal/original/2X/7/76f8fa7a05992bfe4ac545ff43dddca46cd4e43c.png)

---

<div class="post-metadata">

**Author:** ![avarabyeu](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/avarabyeu/32/3846_2.png) [@avarabyeu](https://community.temporal.io/u/avarabyeu)\
**Post date:** [May 30, 2023, 10:11pm UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385/7 "2023-05-30T22:11:40Z")

</div>

Can confirm - UI and temporal CLI (as well as API using golang SDK) show different list of attributes. Custom attributes seem to be created successfully over CLI or API, but adding them to workflow results in `Namespace default has no mapping defined for search attribute <attribute name>`  
Temporal CLI 0.9.0, SQLLite persistance.  
Attempting to set/unset `system.forceSearchAttributesCacheRefreshOnRead` has no affect

---

<div class="post-metadata">

**Author:** ![petersamokhin](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/petersamokhin/32/3848_2.png) [@petersamokhin](https://community.temporal.io/u/petersamokhin)\
**Post date:** [May 30, 2023, 10:31pm UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385/8 "2023-05-30T22:31:57Z")

</div>

Hello there! I face the same issue with typescript SDK (`upsertSearchAttributes`) & typescript worker, while creating the attributes correctly and seeing them in the temporal web UI and using cli (`temporal operator search-attribute list`).

```auto
{
  "message": "BadSearchAttributes: Namespace MY_NAMESPACE has no mapping defined for search attribute MY_SEARCH_ATTRIBUTE",
  "source": "",
  "stackTrace": "",
  "encodedAttributes": null,
  "cause": null,
  "serverFailureInfo": {
    "nonRetryable": true
  }
}

```

Binary Checksum: `@temporalio/worker@1.7.4+e7abfc11f1d9da4c176cc2b7ca184b1e81106f264157a84eaa4801078763688a`

---

<div class="post-metadata">

**Author:** ![taonic](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/taonic/32/5556_2.png) [@taonic](https://community.temporal.io/u/taonic)\
**Post date:** [May 31, 2023, 7:15am UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385/9 "2023-05-31T07:15:57Z")

</div>

@karney Thanks for pointing out the misleading env var names in CLI’s prompt. ~~I’ll follow up it from my side.~~ This has been fixed [here](https://github.com/temporalio/cli/pull/239) and should be included in the next release with updated document too.

@avarabyeu and @petersamokhin

Firstly, I’d like to acknowledge the defect in the UI where the custom search attributes are not showing up accordingly:

> <https://github.com/temporalio/ui/issues/1274>
>
> \### Describe the bug
> 
> Custom search attributes do not show up in UI, even thou…gh they do appears in CLI.
> 
> 
> \### Minimal Reproduction
> 
> 1. Start a dev server:
> 
> \`\`\`
> temporal server start-dev
> \`\`\`
> 
> 2. From a different terminal, add a custom attribute.
> 
> \`\`\`
> temporal operator search-attribute create --namespace default --name CustomAttr1 --type Text --yes
> sleep 10
> 
> temporal operator search-attribute list --namespace default
> \# Observe that CustomAttr1 appears in the list of attributes
> \`\`\`
> 
> 3. Visit \`http://127.0.0.1:8233/namespaces/default\`, and observe that \`CustomAttr1\` is not listed.
> 
> \### Environment/Versions
> 
> \- Tested with CLI 0.8.0, but investigation suggests that this is independent of CLI

Now, based on what Karney described above, as well as my earlier test, the error message such as `Namespace default has no mapping defined for search attribute <attribute name>` may have something to do with different namespaces being used while creating the custom attribute and consuming/updating it. Note, custom attributes are scoped to namespaces. So it’d be great if you could help me to double check this scenario first. Thanks!

---

<div class="post-metadata">

**Author:** ![avarabyeu](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/avarabyeu/32/3846_2.png) [@avarabyeu](https://community.temporal.io/u/avarabyeu)\
**Post date:** [May 31, 2023, 8:40am UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385/10 "2023-05-31T08:40:22Z")

</div>

I tend to think that issue is related to back-end, not to UI. Mostly, because we cant create workflow with just created search attributes (as well as upsert them to the workflow).  
As far namespace part - the UI says i’ve got only “default”. I see “default” and “temporal-system” in DB and all custom attributes are created with passing “default” explicitly.  
Is there anything else i can check in SQLLite to help you with figuring out?

---

<div class="post-metadata">

**Author:** ![taonic](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/taonic/32/5556_2.png) [@taonic](https://community.temporal.io/u/taonic)\
**Post date:** [May 31, 2023, 10:07am UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385/11 "2023-05-31T10:07:41Z")

</div>

Let’s eliminate the SDK for now. Can you share the result of these cli commands? e.g.

```auto
$ temporal operator search-attribute create --name MyKeyword1 --type Keyword
Search attributes have been added

$ temporal workflow start --type test --task-queue test --search-attribute MyKeyword1=\"abc\
Running execution:
  WorkflowId 95cd7128-1b3a-49b3-af65-edf2532a83c5
  RunId f6660b6a-8dd3-4a72-8440-7486cc54f3e0
  Type test
  Namespace default
  TaskQueue test
  Args []

$ temporal -v
temporal version 0.9.0 (server 1.20.2) (ui 2.15.0)

```

FWIW, this is my server command:

```auto
temporal server start-dev -db-filename db

```

---

<div class="post-metadata">

**Author:** ![karney](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/karney/32/3032_2.png) [@karney](https://community.temporal.io/u/karney)\
**Post date:** [May 31, 2023, 11:31am UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385/12 "2023-05-31T11:31:44Z")

</div>

I was able to register the values in the non-default namespace after specifying it explicitly using the namespace option in the command. The initial problem is likely because of the env var not having any effect.

---

<div class="post-metadata">

**Author:** ![avarabyeu](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/avarabyeu/32/3846_2.png) [@avarabyeu](https://community.temporal.io/u/avarabyeu)\
**Post date:** [May 31, 2023, 11:33am UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385/13 "2023-05-31T11:33:19Z")

</div>

`Search attributes have been added`

```auto
Running execution:
  WorkflowId 2d471a2b-8b6a-4e65-af58-0f2e9c757fcc
  RunId 270c2058-2fb3-4291-af69-883218aba5e1
  Type test
  Namespace default
  TaskQueue test
  Args []

```

so it works perfectly well. now it gets a bit interesting. I tried to do the same with my code (adding an attribute via temporal client and then starting the workflow) and it doesn’t work. Probably, something is related to caching or asynchronous nature of adding new search attribute. The temporal is run as:  
`temporal server start-dev -n default -db-filename db --dynamic-config-value system.forceSearchAttributesCacheRefreshOnRead=true`

---

<div class="post-metadata">

**Author:** ![avarabyeu](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/avarabyeu/32/3846_2.png) [@avarabyeu](https://community.temporal.io/u/avarabyeu)\
**Post date:** [May 31, 2023, 11:59am UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385/14 "2023-05-31T11:59:31Z")

</div>

Well, it looks like there are two separate issues - with UI showing incorrect list of attributes and on my end with caches.  
So, my sequence of actions:

1. Start the dev server, wait for cluster to become healthy
2. Add new search attributes over client SDK
3. Execute list attributes, make sure new attributes are returned
4. Start new workflow that uses just created attributes (also over client SDK)  
As result, search attributes aren’t available for the workflow and the result is `Namespace default has no mapping defined for search attribute`)  
Still, if i add some 20 secs wait in between of (3) and (4) - it works perfectly fine. The same reproduces regardless the value `system.forceSearchAttributesCacheRefreshOnRead` set to

---

<div class="post-metadata">

**Author:** ![avarabyeu](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/avarabyeu/32/3846_2.png) [@avarabyeu](https://community.temporal.io/u/avarabyeu)\
**Post date:** [May 31, 2023, 4:22pm UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385/15 "2023-05-31T16:22:35Z")

</div>

@taonic Is there any way to make sure that custom search attributes are available for use by ExecuteWorkflow? There is still some time in between of `ListSearchAttributes` returns new attributes and `ExecuteWorkflow` stops responding with the error that attribute isn’t defined

---

<div class="post-metadata">

**Author:** ![taonic](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/taonic/32/5556_2.png) [@taonic](https://community.temporal.io/u/taonic)\
**Post date:** [June 2, 2023, 12:54pm UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385/16 "2023-06-02T12:54:20Z")

</div>

Turns out it’s a regression: [Regression: namespace specific search attributes have registration delay with `system.forceSearchAttributesCacheRefreshOnRead=true` · Issue #4017 · temporalio/temporal · GitHub](https://github.com/temporalio/temporal/issues/4017)

---

<div class="post-metadata">

**Author:** ![avarabyeu](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/avarabyeu/32/3846_2.png) [@avarabyeu](https://community.temporal.io/u/avarabyeu)\
**Post date:** [June 12, 2023, 6:04pm UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385/17 "2023-06-12T18:04:58Z")

</div>

@taonic thank you for sharing the update on this!

---

<div class="post-metadata">

**Author:** ![CliffSun](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/cliffsun/32/1488_2.png) [@CliffSun](https://community.temporal.io/u/CliffSun)\
**Post date:** [August 18, 2023, 1:04pm UTC](https://community.temporal.io/t/custom-search-attribute-doesnt-work-with-temporal-server-start-dev/8385/18 "2023-08-18T13:04:05Z")

</div>

Am I right in saying that there’s still an issue with displaying the custom search attributes in the UI? This is what I’ve still noticed when starting a temporal dev server using the latest Temporal CLI. The actual search attribute is registered and creating workflows that set that search attribute succeeds, so it sounds like a bug on the UI side.