Custom serialization in Go

Community Support
1

Hi,

I am trying to pass a struct of strings in my client.ExecuteWorkflow call in the workflow starter. However, it does not seem to serialize/deserialize properly as the fields of the struct are empty when the workflow starts and reads that struct input parameter.

From looking at the code, this seems related to specifying a dataConverter, but I am not able to crack this so far.

Thank you for your help,
Jacques

2

The Go JSON serializer skips private fields. Try changing their visibility to public.

1 Like
3

My fields are public. Is there anything else I could try? The struct only contains strings, as follows

type Params struct {
Param1 string
…
}

4

What is the value of input field of the WorkflowExecutionStartedEvent?

And what is the signature of the workflow function?

5

I was not reading my own logs properly. The JSON serializer is working fine. Thank you Maxim for the help.

1 Like
6

Hi @maxim,

Resurrecting this ticket to ask the same question. We’re using protobufs but are seeing that any struct with a OneOf field is able to encode without issue, but is not able to be decoded by temporal. Can we override the default JSON encoder library? If so I’d like to use jsonpb as our encoder. jsonpb · pkg.go.dev

Error Message


  "message": "payload item 0: unable to decode: json: cannot unmarshal object into Go struct field Redacted.oneOfProp.OneOfValue of type redacted.isRedacted_OneOfValue",
  "source": "GoSDK",
  "stackTrace": "",
  "cause": {
    "message": "unable to decode: json: cannot unmarshal object into Go struct field Redacted.oneOfProp.OneOfValue of type redacted.isRedacted_OneOfValue",
    "source": "GoSDK",
    "stackTrace": "",
    "cause": {
      "message": "unable to decode",
      "source": "GoSDK",
      "stackTrace": "",
      "cause": null,
      "applicationFailureInfo": {
        "type": "",
        "nonRetryable": false,
        "details": null
      },
      "failureInfo": "applicationFailureInfo"
    },
    "applicationFailureInfo": {
      "type": "wrapError",
      "nonRetryable": false,
      "details": null
    },
    "failureInfo": "applicationFailureInfo"
  },
  "applicationFailureInfo": {
    "type": "wrapError",
    "nonRetryable": false,
    "details": null
  },
  "failureInfo": "applicationFailureInfo"
}
7

Our ProtoJsonPayloadConverter unit tests test structures that have oneof fields.

Could you look at the test and find out what is different in your case?

8

Thanks for the link I’ll check it out.

9

@maxim Looks the generated code is mostly the same. Only difference is that the internal value is a pointer (which is non-nil in my data).

So in the equivalent of this instantiation, the ValueS: "awd" would be a pointer to a struct like ValueStruct: &Struct{} and here it’s generated to be a pointer instead of a direct value.

Slight update: I did some proof of my theory. That wasn’t the issue (listed above). The issue was that I was returning it in an array. But, returning it in an array should work… right?

10

Slight update: I did some proof of my theory. That wasn’t the issue (listed above). The issue was that I was returning it in an array. But, returning it in an array should work… right?

I think I understand the problem.

The DataConverter picks the PayloadConverter based on the type of a value. In the case of the array it picks JSONPayloadConverter as the array is not of a protobuf type. The JSONPayloadConverter internally relies on golang “encoding/json” package. This package has problems dealing with Protobufs in edge cases like one-of.

When the top level object is a protobuf then the ProtoJSONPayloadConverter is selected and serialization works fine.

So the solution in your case is to pass as an argument an enclosing protobuf that contains the repeated field of the proto you currently store in the array.

1 Like
11

@maxim we are facing the same issue with our temporal SDK. Do I need to pass it separately as a separate struct?

we use a lot of oneof in our apis. I really don’t want to change signatures.

options := workflow.WithActivityOptions(workflow.WithDataConverter(ctx,
		temporalConvertor.NewCompositeDataConverter(
			temporalConvertor.NewNilPayloadConverter(),
			temporalConvertor.NewByteSlicePayloadConverter(),

			// Order is important here. Both ProtoJsonPayload and ProtoPayload converters check for the same proto.Message
			// interface. The first match (ProtoJsonPayload in this case) will always be used for serialization.
			// Deserialization is controlled by metadata, therefore both converters can deserialize corresponding data format
			// (JSON or binary proto).
			temporalConvertor.NewProtoJSONPayloadConverter(),
			temporalConvertor.NewProtoPayloadConverter())),
		workflow.ActivityOptions{
			StartToCloseTimeout: 90 * time.Second,
			RetryPolicy: &temporalInternal.RetryPolicy{
				MaximumAttempts: 2,
			},
		})

it still doesn’t work.

12

Sorry, I don’t understand the question. Would you open a new topic with your question?