# \[Error in UI\]13 INTERNAL: ListClosedWorkflowExecutions failed. Error: elastic: Error 503 (Service Unavailable): all shards failed

**URL:** <https://community.temporal.io/t/error-in-ui-13-internal-listclosedworkflowexecutions-failed-error-elastic-error-503-service-unavailable-all-shards-failed/2351>\
**Category:** Community Support\
**Tags:** elasticsearch, mysql, database\
**Created:** [June 10, 2021, 7:54pm UTC](https://community.temporal.io/t/error-in-ui-13-internal-listclosedworkflowexecutions-failed-error-elastic-error-503-service-unavailable-all-shards-failed/2351 "2021-06-10T19:54:46Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![mohits](https://avatars.discourse-cdn.com/v4/letter/m/bbe5ce/32.png) [@mohits](https://community.temporal.io/u/mohits)\
**Post date:** [June 10, 2021, 7:54pm UTC](https://community.temporal.io/t/error-in-ui-13-internal-listclosedworkflowexecutions-failed-error-elastic-error-503-service-unavailable-all-shards-failed/2351/1 "2021-06-10T19:54:46Z")

</div>

![image](https://us1.discourse-cdn.com/flex016/uploads/temporal/original/2X/c/c2b82520e2cf6567cfc883725aabf6e50cb41cca.png)

13 INTERNAL: ListClosedWorkflowExecutions failed. Error: elastic: Error 503 (Service Unavailable): all shards failed [type=search\_phase\_execution\_exception]

temporal frontend logs :: `{"level":"error","ts":"2021-06-10T19:44:27.710Z","msg":"Operation failed with internal error.","service":"frontend","metric-scope":240,"error":"ListClosedWorkflowExecutions failed. Error: elastic: Error 503 (Service Unavailable): all shards failed [type=search_phase_execution_exception]","logging-call-at":"esVisibilityMetricClients.go:332","stacktrace":"go.temporal.io/server/common/log/loggerimpl.(*loggerImpl).Error\n\t/temporal/common/log/loggerimpl/logger.go:138\ngo.temporal.io/server/common/persistence/elasticsearch.(*visibilityMetricsClient).updateErrorMetric\n\t/temporal/common/persistence/elasticsearch/esVisibilityMetricClients.go:332\ngo.temporal.io/server/common/persistence/elasticsearch.(*visibilityMetricsClient).ListClosedWorkflowExecutions\n\t/temporal/common/persistence/elasticsearch/esVisibilityMetricClients.go:163\ngo.temporal.io/server/common/persistence.(*visibilityManagerWrapper).ListClosedWorkflowExecutions\n\t/temporal/common/persistence/visibilityWrapper.go:159\ngo.temporal.io/server/service/frontend.(*WorkflowHandler).ListClosedWorkflowExecutions\n\t/temporal/service/frontend/workflowHandler.go:2550\ngo.temporal.io/server/service/frontend.(*DCRedirectionHandlerImpl).ListClosedWorkflowExecutions.func2\n\t/temporal/service/frontend/dcRedirectionHandler.go:337\ngo.temporal.io/server/service/frontend.(*NoopRedirectionPolicy).WithNamespaceRedirect\n\t/temporal/service/frontend/dcRedirectionPolicy.go:116\ngo.temporal.io/server/service/frontend.(*DCRedirectionHandlerImpl).ListClosedWorkflowExecutions\n\t/temporal/service/frontend/dcRedirectionHandler.go:333\ngo.temporal.io/api/workflowservice/v1._WorkflowService_ListClosedWorkflowExecutions_Handler.func1\n\t/go/pkg/mod/go.temporal.io/api@v1.4.0/workflowservice/v1/service.pb.go:1407\ngo.temporal.io/server/common/authorization.(*interceptor).Interceptor\n\t/temporal/common/authorization/interceptor.go:136\ngoogle.golang.org/grpc.getChainUnaryHandler.func1\n\t/go/pkg/mod/google.golang.org/grpc@v1.34.0/server.go:1051\ngo.temporal.io/server/common/rpc.ServiceErrorInterceptor\n\t/temporal/common/rpc/grpc.go:100\ngoogle.golang.org/grpc.chainUnaryServerInterceptors.func1\n\t/go/pkg/mod/google.golang.org/grpc@v1.34.0/server.go:1037\ngo.temporal.io/api/workflowservice/v1._WorkflowService_ListClosedWorkflowExecutions_Handler\n\t/go/pkg/mod/go.temporal.io/api@v1.4.0/workflowservice/v1/service.pb.go:1409\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.34.0/server.go:1210\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.34.0/server.go:1533\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.34.0/server.go:871"}`

I am seeing this error in UI and couldn’t see much information.Can you suggest me on this error ?

---

<div class="post-metadata">

**Author:** ![alex](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/alex/32/15_2.png) [@alex](https://community.temporal.io/u/alex)\
**Post date:** [June 10, 2021, 7:58pm UTC](https://community.temporal.io/t/error-in-ui-13-internal-listclosedworkflowexecutions-failed-error-elastic-error-503-service-unavailable-all-shards-failed/2351/2 "2021-06-10T19:58:00Z")

</div>

How did you setup your Elasticsearch? Did you create index template and index prior running Temporal Server?

---

<div class="post-metadata">

**Author:** ![mohits](https://avatars.discourse-cdn.com/v4/letter/m/bbe5ce/32.png) [@mohits](https://community.temporal.io/u/mohits)\
**Post date:** [June 10, 2021, 8:40pm UTC](https://community.temporal.io/t/error-in-ui-13-internal-listclosedworkflowexecutions-failed-error-elastic-error-503-service-unavailable-all-shards-failed/2351/3 "2021-06-10T20:40:57Z")

</div>

We have used this command to deploy temporal server.

`helm install -f values/values.mysql.yaml temporaltest . --namespace live-staging --timeout 900s`

and we are using our own mysql server.

It used to work before where we saw the workflows in UI, but all of the sudden we see this issue.  
There is no change on the temporal deployment part from where we saw this issue.

---

<div class="post-metadata">

**Author:** ![alex](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/alex/32/15_2.png) [@alex](https://community.temporal.io/u/alex)\
**Post date:** [June 10, 2021, 9:22pm UTC](https://community.temporal.io/t/error-in-ui-13-internal-listclosedworkflowexecutions-failed-error-elastic-error-503-service-unavailable-all-shards-failed/2351/4 "2021-06-10T21:22:18Z")

</div>

With this command you will have your own Elasticsearch deployed within a Temporal cluster. Which is not good nor for prod neither for staging. Apparently your cluster restarted and you’ve lost your Elasticsearch index. Do you really need [enhanced visibility](https://docs.temporal.io/docs/server/elasticsearch-setup) (this is what Elasticsearch for)? If yes, I would recommend you to install Elasticsearch outside of your Temporal cluster (same way as you do for MySQL), configure it in `values/values.elasticsearch.yaml` file and pass it to the `helm` command. If not, just simply pass `--set elasticsearch.enabled=false` to the `helm` command and MySQL will be used for basic visibility.

---

<div class="post-metadata">

**Author:** ![mohits](https://avatars.discourse-cdn.com/v4/letter/m/bbe5ce/32.png) [@mohits](https://community.temporal.io/u/mohits)\
**Post date:** [June 11, 2021, 4:44am UTC](https://community.temporal.io/t/error-in-ui-13-internal-listclosedworkflowexecutions-failed-error-elastic-error-503-service-unavailable-all-shards-failed/2351/5 "2021-06-11T04:44:25Z")

</div>

Now we have redeployed the temporal server with `--set elasticsearch.enabled=false`  
and we are seeing different error.  
`13 INTERNAL: ListClosedWorkflowExecutions failed. Error: elastic: Error 400 (Bad Request): all shards failed [type=search_phase_execution_exception]`

 ![image](https://us1.discourse-cdn.com/flex016/uploads/temporal/original/2X/4/401e9c62a65870685ddc2b5a925d0a75515f1364.png)

Please let us know if we are doing something wrong here.

---

<div class="post-metadata">

**Author:** ![alex](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/alex/32/15_2.png) [@alex](https://community.temporal.io/u/alex)\
**Post date:** [June 11, 2021, 7:08pm UTC](https://community.temporal.io/t/error-in-ui-13-internal-listclosedworkflowexecutions-failed-error-elastic-error-503-service-unavailable-all-shards-failed/2351/6 "2021-06-11T19:08:18Z")

</div>

Standart visibility uses another MySQL database: [temporal\_visibility](https://github.com/temporalio/helm-charts/blob/master/values/values.mysql.yaml#L24). It seems that you have configured visibility to use the same `temporal` database. But anyway you need to create visibility [tables](https://github.com/temporalio/temporal/blob/master/schema/mysql/v57/visibility/schema.sql) in the configured database.
