Potential impact of workflow update

Community Support
1

Hi,
I have a question regarding what potential impact it causes when updating a workflow definition.

Say, I have a workflow with version V0, and there is 1 running instance R0. Now I add an activity so the workflow version becomes V1 and then I restart the workflow service (where the workflow gets defined and running).

  1. will R0 be executed based on workflow version V0 or V1?
  2. for new triggered workflow instance, will it be executed based on V0 or V1?
  3. is there any issue for doing this: updating workflow definition and just restart the service

Looks like it is logically incorrect if R0 will be executed based on V1. So a generic question: is there a solution for workflow live update and version management, so that the workflow running instances for different versions can be isolated? If not, what would I do if I want to update the workflow definition when we still have some ongoing workflow instances.

2

Temporal doesn’t version the whole workflow. So the question of which workflow version is going to run V0 or V1 is not applicable.

Temporal versions any part of the workflow code independently. So if an activity is added only that part of code will be versioned and both codepaths the old and the new will be present in the code. The new code path will be always taken for the newly executed code and the code path that was used to execute code initially will be used when replaying workflow from the event history.

This model allows deploying the updated workflow at any time and restarting the service.

See getVersion documentation for Go and Java SDKs for details.

3

For some use cases, we don’t want the old running instance to execute the newly added activity. For example, we added a new payment method but the old users are not compatible with the new payment method etc…

If this is the case, how should I update the workflow? should i create a copy of old workflow then add the activity to the copied one?

4

GetVersion API first parameter is changeId. All calls to GetVersion that share the same changeId always return the same version number. So if you add a call to GetVersion at the beginning of your workflow and then another call with the same changeId when new activity is added all already executing workflows will use the old version. This will happen because the place of the first GetVersion call was already executed by all these workflows.

5

but the workflow code logic has already had the new activity. If using the old changeId to getVersion at the beginning of the workflow, will the execution just skip the unrecognized activity? basically do nothing and return a nil err

6

just read the comment of getVersion call, the version has to be checked to decide what to do for different version. So there is no such issue i mentioned before existed

1 Like
7

I saw this
// It is not allowed to update workflow code while there are workflows running as it is going to break
// determinism.

seems that having the getVersion mechanism, we can update a workflow code while there are workflows running with no issues, correct?

8

Correct, we need to make the documentation more clear that GetVersion is the supported way for updating workflow code for running workflows.

9

want to make sure i understand correctly, for this example here:

workflow v0:

err = workflow.ExecuteActivity(ctx, foo).Get(ctx, nil)

workflow v1:

v :=  GetVersion(ctx, "fooChange", DefaultVersion, 1)
if v  == DefaultVersion {
    err = workflow.ExecuteActivity(ctx, foo).Get(ctx, nil)
} else {
    err = workflow.ExecuteActivity(ctx, bar).Get(ctx, nil)
}

after changing the workflow to V1, for new workflow and old workflow which has not hit there yet, it will return version number 1 (correct?). To replay finished workflow or workflow which has passed there, what it will be return?

10

after changing the workflow to V1, for new workflow and old workflow which has not hit there yet, it will return version number 1 (correct?).

Yes, correct.

To replay finished workflow or workflow which has passed there, what it will be return?

It returns the value that was returned the first time or DefaultVersion if GetVersion wasn’t there when the code was executed for the first time.

11

you mentioned before that:
All calls to GetVersion that share the same changeId always return the same version number.

when i replay a workflow, and pass changeID as fooChange, wouldn’t it return version 1?

12

One big assumption that I forgot to mention is that all GetVersion calls that share changeId must be added together as they represent a single change across the workflow code.

So if you add GerVersion with fooChange at the beginning of the workflow and at the point where workflow wasn’t executed yet, both with return DefaultVersion as the beginning of the workflow was already executed.

13

I still got some questions:

  1. how does temporal server know the changeID? is it passed through some APIs?
  2. for V0, i dont have any getVersion call, when I change to V1, should I add getVersion calls to the beginning and the place where the change was made?

can you show a complete example for this? the example in the comment is not quite understandable

14
  1. I’m not sure I understand the question. GetVersion accepts the changeID as the first parameter. It is recorded into MarkerRecordedEvent which the server never looks into as it is used only by SDK code.
  2. No, you don’t need to add GetVersion anywhere beyond places where code was changed. The only reason I mentioned the beginning of the workflow because of the stated requirement (which sounds really weird to me) that if workflow already started the new version shouldn’t be used.
15

for question 1: I meant where is this changeID coming from, is it named by user who change workflow definition or it is auto-named by SDK and should be looked up through a API something

changeId is named by the user that is updating workflow definition. Each update must have a different changeId. If update spawns multiple places in the code all the GetVersion calls for the same update should share changeId.

for question 2: a simple example for the requirement: say we have a new payment method adopted, so we change workflow definition to add an activity for it, for all ongoing workflow running instances, we dont want them to hit that payment method, since it is not supported for them. So we only want the new instances to execute the new payment activity. I understand getVersion(“new_payment_change_id”) will return the version number V1 for the new instances, but for the ongoing instances when it hit the getVersion what will be returned? since the change id is also “new_payment_change_id”

If you just add new activity protected a single GetVersion("new_payment_change_id") call then any workflow including already running is going to get V1 when reaching this statement for the first time.

But as I mentioned before if you also add GetVersion("new_payment_change_id") at the beginning of the workflow both calls to GetVersion are going to return DefaultVersion. The reason is that all calls to GetVersion with the same changeId are guaranteed to return the same value . And as the GetVersion at the beginning of the workflow was added to already executed portion of the workflow it must return DefaultVersion.

16

  1. so the way to update workflow is:
    a. change the workflow definition code
    b. add getVersion calls with a changeID, what if we add getVersion call with new changeID but remain the workflow definition as before?

  2. this seems not to solve my problem. my requirement is that finished and ongoing workflows wont hit the new payment method, all new workflow will hit the new payment.
    will the following work? I dont have getVersion call for v0

// workflow version 0
err = workflow.ExecuteActivity(ctx, CreditCardPay).Get(ctx, nil)

// workflow version 1
v := GetVersion(ctx, “add_paypal”, DefaultVersion, 1)
if v == DefaultVersion {
err = workflow.ExecuteActivity(ctx, CreditCardPay).Get(ctx, nil)
} else {
err = workflow.ExecuteActivity(ctx, PaypalPay).Get(ctx, nil)
}

17
  1. so the way to update workflow is:
    a. change the workflow definition code
    b. add getVersion calls with a changeID, what if we add getVersion call with new changeID but remain the workflow definition as before?

You don’t just add calls to GetVersion. You usually use the result of it to switch between different versions of the workflow code. So if you add GetVersion and don’t change the code there is no harm.

  1. this seems not to solve my problem. my requirement is that finished and ongoing workflows wont hit the new payment method, all new workflow will hit the new payment.
    will the following work? I dont have getVersion call for v0

It solves your problem if you add GetVersion call at the beginning of the workflow. This way new started workflows are going to see v==1 and all already executing workflows will see the DefaultVersion.

fund MyWorkflow(...) ... {
    // Make sure that already running workflows will use DefaultVersion
    GetVersion("add_paypal", 1) // result is ignored here
    ...
    // at the end of the workflow
    v := GetVersion(ctx, “add_paypal”, DefaultVersion, 1)
    if v == DefaultVersion {
        err = workflow.ExecuteActivity(ctx, CreditCardPay).Get(ctx, nil)
    } else {
        err = workflow.ExecuteActivity(ctx, PaypalPay).Get(ctx, nil)
    }
18

gotcha, but this seems to be a little subtle, and not very understandable just by reading the code for ppl who are not familiar with temporal

ideally, if just adding one getVersion right before the new activity, and temporal server can return corresponding supposed version for different workflow instances, that would be good. Not sure if that is feasible

19

Your use case is pretty exceptional. The majority of users want to version every part of the workflow independently.

20

is it exceptional? from the business point of view, different workflow instances should correspond to different workflow versions right? Isnt the goal of versioning different part of the workflow to execute different logic for different workflow instances?