Temporal Activity Schema Change Backwards Compatibility

Community Support
, ,
1

I am attempting to implement a change to my Activity request schema and wanted to test the backwards compatibility using replay tests in order to determine if I need to incorporate version based branching in my workflow code.

Looking at the java and go versioning documentation, it’s not always clear what changes need to be versioned with branching logic. I assumed that any change to the workflow behavior or activity req/res schema would break the deserialization of the workflow execution state.

To test such changes, I leveraged replay tests using the java test sdk (1.4), which I expected to give me relatively high confidence that my change will not be breaking the older executions of workflows.

In my current example, I am changing the type of a field on the ActivityInterface request schema, which can be serialized in the same format. For example, UUID to String. The serialization for these two values can be the same. From what I can tell, the workflow execution state serialization does not include any type information, so I would hope that this change is backwards compatible. The replay test did not fail in this case.

To test out the replay functionality, I played around with the schema more and added/removed fields from the request and the response schemas (4 more tests for a total of 5). The only failure was a removal of a field from the activity response, which wasn’t even read in the workflow.

I could also test this out myself, but if anyone else is already aware of the expected behavior of Temporal for these cases, that would be helpful.

2

I stumbled upon this slack thread, which indicated that a best practice is to use a single struct for req/res payloads.

Will keep that in mind moving forward, but I still have the same complication of migrating to that point.

3

Do you have running workflow executions for the workflow type you are trying add changes to?

4

Yes. I just ran some tests with a local temporal instance and arrived at the following conclusions. Please let me know if there are any misconceptions:

  • When an activity task is scheduled, a json payload is serialized based on the input parameters for the method signature for the current version.
  • The framework will attempt to deserialize the activity request payload from the execution state when the scheduled activity task is taken off the queue and the method is invoked. In java, it’s deserialized by jackson.
  • If an activity method signature (java) is changed (new version) while executions are pending the following will occur (due to jackson behavior, probably):
    • new fields on the signature that are missing from the json payload will be set to default values
    • removed fields from the signature that are present in the json payload will be ignored
    • fields whose type are modified will have behavior depending on how the parser can serialize them. For example, a UUID type can be changed to the string for jackson, and the activity will run as intended.
5

The default DataConverter for Java SDK uses JacksonJsonPayloadConverter to serialize Java types.
You can plug in your custom data converter if needed.

Activity task inputs (as well as results) are recorded in the workflow history (see the ActivityTaskScheduled events for example). During history replay it should be possible to deserialize the recorded inputs according to the used activity method signature.

You should be able to add and remove input params at the end of the activity method signature without breaking determinism. The extra ones at the end are as you mentioned, going to have default values.
Note that the activity implementation itself can change any time without breaking determinism.

I think in your case you should be able to add the extra parameters to the end of the activity signature, and change the activity implementation to use the new ones instead of old (if you don’t want to use versioning).

1 Like