Protos in activities arguments

Support
#1

Hello,

We were updating temporal version SDK from 0.27 to 0.29 but had issues with arguments passed to activities. Seems that passing pointers of proto messages with enums is not able to encode / decode.

protoc versions for ref:

protoc-gen-go v1.20.0
protoc v3.11.4

Seems the ProtoJSONPayloadConverter used is using jsonpb ( https://godoc.org/github.com/golang/protobuf/jsonpb ) which is deprecated in favor of protojson (https://godoc.org/google.golang.org/protobuf/encoding/protojson )

We’ve created a Converter to use protojson and seems to be working for now. Couldn’t find any issues on github about this. If protojson support will be added, please inform the ticket so we can revert to the default DataConvert again.

Best regards,
Pedro Almeida

1 Like
#3

Hi Pedro.
First of all, we don’t use github.com/golang/protobuf/jsonpb and if you follow “go to declaration” links in Go SDK you will see that github.com/gogo/protobuf/jsonpb is actually used as all our go wrappers for proto are generated using gogo. I did some performance comparison and according to it and overall convenience that gogo provides we don’t plan to switch to google version of protobuf in near future.
I also added unit test to test how enums are serialized inside proto objects. They are passed and enums seems to work fine.
My guess would be that you use google/progobuf library to generate your go proto objects and they can’t be serialized with our gogo based serializer. Can you confirm this?

#4

Hello Alex,

Thanks for the reply. Indeed, i didn’t noticed the gogo in the path. My bad.

We generate our proto objects with protoc-gen-go version 1.20.
( https://github.com/protocolbuffers/protobuf-go/releases/tag/v1.20.0 )

Not sure this answer your question. Not familiar with google/progobuf.

We have this on the top of the generated objects :

// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.20.0
// protoc v3.11.4

Is this enough?

#5

Yes, this is google version and apparently structs are generated by it is not supported by gogo serializer. I see two possible solutions here:

  1. Create your custom payload converter and build your custom data converter using NewCompositeDataConverter: https://github.com/temporalio/go-sdk/blob/00fb03096c85ee092304125c0054f2d78de377a8/converter/default_data_converter.go#L28 (apparently this is what you did).
  2. Switch to gogo to generate your go code.

We might add support for google version later.

#6

Hi Alex,

Thanks for looking into it.

Indeed, we created a custom payload Converter and for now seems to be working fine.

If we have issues, will eval gogo as suggested.

Thanks again.

#7

Hi Pedro.

I have some update on this. After exploring different approaches I came up with solution which supports both gogo and standard google serializer.
This PR adds support for the structs generated with standard google plugin v1.4.0 and above (including new APIv2). It should be supported for both JSON and proto serialization without any configurations on user side.

#8

Hello Alex.

Thanks for the info. Will remove the CustomDataConverter from our solution and do some tests.

Thanks again for all the help.

#9

@Pedro_Almeida @alex I found this PR already in the released v1.3.0 sdk, but I still this error:

“GetStatus activity failed” {err 15 0 payload item 0: unable to decode: unknown value “New” for enum package.Status }=""

Is my protoc-gen-go simply too old?

#10

Why not use latest? But I think it is not related. What are the values of package.Status? This error comes from protojson.Unmarshal and it means that JSON can’t be deserialized back to struct.

#11

It just so happens that that was the issue exactly. The binary was from 2019 December, updating it and recompiling everything worked like a charm.

2 Likes