we are upgrading our temporal install from 1.13.3 to 1.16.2. I have found in the release notes for 1.15.0 that there is this change which updates the index settings for elasticsearch.
Question: Are these changes required for the upgrade? We have a standard process in place to upgrade our database schema using temporal-cassandra-tool … however we haven’t made any ES updates since we added it (which was after 1.13.3 for us).
If we do need to make these index settings changes, can you provide some guidance on how to do so? If it is not needed please let us know as well. Thank you!
Those changes are not required. They were made to optimize Elasticsearch performance. If default page of web UI works fine for you then you don’t need it. We did load tests and found out that with some large number of workflows (1M+) default web UI page may experience some slowness. If you have same experience then these Elasticsearch schema changes will help you.
Unfortunately, we don’t have automation tool to do Elasticsearch schema changes. Basically it requires some steps similar to what we did back in v1.11.0.
Thanks for the response @alex. That resolves our question for now, we’re going to proceed w/ the upgrad w/out the schema change to ES. However are queueing up the work to make the index change per the v1.11.0 release notes process in order to get those performance gains in the future. On that note - is Temporal building in a native process for index rotation of ES? It looked like that was the intent of the PR I linked…
Yes, there is ability to enable dual write to two indexes, which you can enable and run for your retention period. If you don’t have long running workflows and retention period is small (I would say less than a month), than you can leverage this approach. It will write to both indexes, and then you just switch read from new one w/o any downtime and reindexing.