# Beginner Question : Using Elastic Search

**URL:** <https://community.temporal.io/t/beginner-question-using-elastic-search/2722>\
**Category:** Community Support\
**Tags:** elasticsearch\
**Created:** [August 14, 2021, 3:05pm UTC](https://community.temporal.io/t/beginner-question-using-elastic-search/2722 "2021-08-14T15:05:02Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![sonyantony](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/sonyantony/32/1209_2.png) [@sonyantony](https://community.temporal.io/u/sonyantony)\
**Post date:** [August 14, 2021, 3:05pm UTC](https://community.temporal.io/t/beginner-question-using-elastic-search/2722/1 "2021-08-14T15:05:02Z")

</div>

Our Project is starting on Temporal.  
It is not clear from the documentation why Elastic search is to be used.

1. Is it being used as long-term storage for the Workflow data?  
( Whenever Workflow data is updated, Temporal sends a POST to Elasticsearch so as to update the ES data )

2. Is a Kafka installation needed for using ES ? ( An old post seems to be suggesting that it is needed: [How to set up ElasticSearch? - #4 by samar](https://community.temporal.io/t/how-to-set-up-elasticsearch/53/4) )

3. Our interest is mainly in the long-term storage of our business objects that will get updated as the result of various workflows. ( Temporal related data and the ability to debug Temporal Workflows is _not_ really important at this point )… is it better if we maintain our business data independent of Temporal-ES integration?

Any help will be much appreciated

---

<div class="post-metadata">

**Author:** ![maxim](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/maxim/32/8_2.png) [@maxim](https://community.temporal.io/u/maxim)\
**Post date:** [August 14, 2021, 8:04pm UTC](https://community.temporal.io/t/beginner-question-using-elastic-search/2722/2 "2021-08-14T20:04:39Z")

</div>

> 1. Is it being used as long-term storage for the Workflow data?  
> ( Whenever Workflow data is updated, Temporal sends a POST to Elasticsearch so as to update the ES data )

No, workflow data is never stored in ES. It only stores so-called “visibility records” which are used to list workflows based on various properties we call “search attributes”.

> 1. Is a Kafka installation needed for using ES ? ( An old post seems to be suggesting that it is needed: [How to set up ElasticSearch? - #4 by samar](https://community.temporal.io/t/how-to-set-up-elasticsearch/53/4) )

It was needed some time ago. The recent releases of Temporal don’t use Kafka for anything.

> 1. Our interest is mainly in the long-term storage of our business objects that will get updated as the result of various workflows. ( Temporal related data and the ability to debug Temporal Workflows is _not_ really important at this point )… is it better if we maintain our business data independent of Temporal-ES integration?

Yes, we recommend storing this business data independently from ES. Another option is to have an always open workflow that maintains that particular entity data. It is hard to give a concrete recommendation without knowing the specifics of your use case.

---

<div class="post-metadata">

**Author:** ![sonyantony](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/sonyantony/32/1209_2.png) [@sonyantony](https://community.temporal.io/u/sonyantony)\
**Post date:** [August 14, 2021, 9:14pm UTC](https://community.temporal.io/t/beginner-question-using-elastic-search/2722/3 "2021-08-14T21:14:43Z")

</div>

Thank you Maxim.  
I thought Temporal will remove the WF data after a ( configurable ) retention period.  
If WF data itself is removed, what use is the ability to search for them from ES ? ( How can we debug if WF data itself is gone ? )

( Sorry like I mentioned, we are just getting started with temporal )

---

<div class="post-metadata">

**Author:** ![maxim](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/maxim/32/8_2.png) [@maxim](https://community.temporal.io/u/maxim)\
**Post date:** [August 16, 2021, 12:07am UTC](https://community.temporal.io/t/beginner-question-using-elastic-search/2722/4 "2021-08-16T00:07:21Z")

</div>

Visibility records are removed from ES after the retention period as well.

---

<div class="post-metadata">

**Author:** ![sonyantony](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/sonyantony/32/1209_2.png) [@sonyantony](https://community.temporal.io/u/sonyantony)\
**Post date:** [August 16, 2021, 1:46pm UTC](https://community.temporal.io/t/beginner-question-using-elastic-search/2722/5 "2021-08-16T13:46:29Z")

</div>

Thank you again Maxim.  
( Sorry ) Im still confused. I thought the very purpose of ES is to act as a long term storage ( along with the ability to run more complicated queries ) .  
But if teh data inside ES will live only as long as teh data inside Temporal, it is not serving as a long term storage.

Could you please point to the Temporal documentation that describes the purpose of ES

---

<div class="post-metadata">

**Author:** ![maxim](https://sea2.discourse-cdn.com/flex016/user_avatar/community.temporal.io/maxim/32/8_2.png) [@maxim](https://community.temporal.io/u/maxim)\
**Post date:** [August 16, 2021, 4:14pm UTC](https://community.temporal.io/t/beginner-question-using-elastic-search/2722/6 "2021-08-16T16:14:24Z")

</div>

Temporal is not designed to act as a long term storage for historical data. ES is used to index open and recently (up to retention period) completed workflows.
