DataConverter Fails to deserialize Thrift Objects -- SOMETIMES in Cadence

Community Support
,
1

Usually the default DataConverter handles serializing/deserializing Thrift objects perfectly fine in cases where the object looks like this:

  class Request {

    int businessId;
    final String name;
    AThriftObject thriftObject;
    List<AThriftObject> thriftObjectList;
}

But the second we try passing this additional field, the default DataConverter throws the exception with stack trace below.

  class Request {
    
    // none of this breaks the DataConverter
    int businessId;
    final String name;
    AThriftObject thriftObject;
    List<AThriftObject> thriftObjectList;

    // the culprit object that breaks the DataConverter
    Map<AThriftObject, String> thriftObjectMap;
}

Is this a bug or definitely unsupported? I noticed the following comment that was a bit unclear to me.

 * Implements conversion through GSON JSON processor. To extend use {@link
 * JsonDataConverter(Function)} constructor. Thrift structures are converted using {@link
 * TJSONProtocol}. When using thrift only one argument of a method is expected.

We are only passing one argument, the Request object, shown above, so it seems like this should work with Thrift.

Thanks for your help!

The default DataConverter throws this exception only if I’m trying to pass a Map<AThriftObject, String>:

Caused by: com.uber.cadence.converter.DataConverterException: Failed to deserialize TBase
	at com.uber.cadence.converter.TBaseTypeAdapterFactory$1.read(TBaseTypeAdapterFactory.java:69)
	at com.google.gson.TypeAdapter$1.read(TypeAdapter.java:199)
	at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:41)
	at com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.read(MapTypeAdapterFactory.java:186)
	at com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.read(MapTypeAdapterFactory.java:145)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:131)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:222)
	at com.google.gson.Gson.fromJson(Gson.java:927)
	at com.google.gson.Gson.fromJson(Gson.java:994)
	at com.uber.cadence.converter.JsonDataConverter.fromDataArray(JsonDataConverter.java:151)
	at com.uber.cadence.internal.sync.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:213)
	at com.uber.cadence.internal.sync.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:190)
	at com.uber.cadence.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:175)
	at com.uber.cadence.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:146)
	at com.uber.cadence.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:71)
	... 3 more
Caused by: org.apache.thrift.protocol.TProtocolException: Unexpected character:D
	at org.apache.thrift.protocol.TJSONProtocol.readJSONSyntaxChar(TJSONProtocol.java:346)
	at org.apache.thrift.protocol.TJSONProtocol.readJSONObjectStart(TJSONProtocol.java:843)
	at org.apache.thrift.protocol.TJSONProtocol.readStructBegin(TJSONProtocol.java:890)
	at com.liveramp.s2s_distribution_lib.soa.DistributionProperty$DistributionPropertyStandardScheme.read(DistributionProperty.java:384)
	at com.liveramp.s2s_distribution_lib.soa.DistributionProperty$DistributionPropertyStandardScheme.read(DistributionProperty.java:379)
	at com.liveramp.s2s_distribution_lib.soa.DistributionProperty.read(DistributionProperty.java:311)
	at org.apache.thrift.TDeserializer.deserialize(TDeserializer.java:81)
	at org.apache.thrift.TDeserializer.deserialize(TDeserializer.java:67)
	at org.apache.thrift.TDeserializer.deserialize(TDeserializer.java:98)
	at com.uber.cadence.converter.TBaseTypeAdapterFactory$1.read(TBaseTypeAdapterFactory.java:66)

Edit: I know we can supply our own custom DataConverter implementation, but I wanted to check to see if the default implementation was supposed to be like this.

2

It looks like gson used by Cadence doesn’t support complex Map object keys out of the box. Would you try to enable it through GsonBuilder.enableComplexMapKeySerialization?

3

Sweet that worked in the test environment like this:


   DataConverter converter = new JsonDataConverter(GsonBuilder::enableComplexMapKeySerialization);
    final Worker worker = testEnv.newWorker(
        TaskLists.DEFAULT_TASK_LIST,
        (workerOptionsBuilder) -> workerOptionsBuilder.setDataConverter(converter)
    );


 final WorkflowClient workflowClient = testEnv.newWorkflowClient(
        new WorkflowClientOptions.Builder()
            .setDataConverter(converter))
            .build()
);

Sounds like it should work the same in production when we instantiate the WorkflowClient to run the main workflow. Thanks Maxim for the quick help!

1 Like