Temporal upgrade - long term server upgrade strategy

Community Support Server Deployment
1

Hi All,
I plan to use Temporal in a setup where upgrades can be scheduled only once in multiple quarters. Therefore, upgrading the temporal itself to each minor version is impossible. Is it possible that upgrade the temporal that skips multiple upgrades?
Based on my limited code reading, it seems like it is possible (based on code in tools/common/schema/updatetask.go). Can you kindly confirm? I am planning to use postgres as DB.

It would be great if you could let me know the long-term server upgrade strategy too.

Regards,
Hemant

2

We have a similar strategy. We just do multiple hop upgrades in the same maintenance. We recently did 1.20 → 1.21 → 1.22 in under an hour.

3

Hi @craigd ,
Thanks for replying and your time.
This upgrade path adds two challenges:

  • Increased payload size: There is a need for all intermediate-release builds to be used.
  • Security and compliance checks/qualifications for each minor release builds

Can you highlight what are the challenges that you might have faced if you upgraded directly from 1.20 → 1.22?

4

Can you highlight what are the challenges that you might have faced if you upgraded directly from 1.20 → 1.22?

There is no guarantee that the Temporal service will function if you skip versions. So you can get any problems up to unrecoverable data corruption.

5

Hi @maxim ,
Can you please point me to the code that executes in the Temporal server after the upgrade and access and/or DB at startup?

What will happen if we invoke the DB update tool for each minor version and update the temporal only once to the desired version?

6

The behavior is undefined. Updating only the DB schema is not enough.

7

Hi @craigd ,
Can you please point me to the code, where DB handling is done at the temporal start? I just want to look at the code for better understanding.

8

I am not that familiar with the codebase for Temporal. You will have to discover for yourself.

9

Hi @craigd ,
Can you please share the commands that you used to upgrade from 1.20 → 1.21 → 1.22?

Thanks

10

Hi @maxim ,
Can you please point me to the code, where DB handling is done at the temporal start? I just want to look at the code for better understanding.

11

We manage our infrastructure on k8s using a gitops methodology and the upgrades are automated via ArgoCD hooks. I expect that is different from how you are doing it.

I would suggest looking at the official documentation to work out an upgrade strategy that works for your specific situation:

12

Hi @maxim,
Sorry for renewing an old thread. But could you please point out the code about this? We are planing to upgrade from 1.21.1 to 1.25.2. I already successfully upgraded directly to 1.25.2 with helm chart 0.52.0. The server work normally (up to now). But I wish to known does everything correct and do not block the upgrade in future.

13

Please don’t upgrade directly from 1.21 to 1.25. There is no guarantee that you will not experience problems with such an upgrade. In the worst-case scenario, you might get corruption that is not recoverable.