How to map Json to Proto using Jackson Mixin?

Question

We are working in an backend application in which we use Protobuffers as model/pojo files. We have to call an API which returns a response as a JSON.

I am looking for a solution to map JSON files directly to proto. java files. As an example, we have Example proto in our project:

message Example{
 string id                    = 1;
 string another_id            = 2;
 int32 code                   = 3;
 string name                 = 4;
}

Now we need to call an API which returns response in JSON:

{
   "json_id":"3",
   "json_another_id":"43",
   "code":34,
   "json_name":"Yeyproto"
}

Now, I want to map the response(which is in json) directly with Proto. please let me know how to do this. Please note since Example.java is an auto generated java file I cannot make any changes in this class. Also, please note the fields of json and proto are different.

Here's what I have tried. I tried to use Jackson Mixin and keep the mapping information in the mixin class but it didnt work and throw some weird FieldDiscriptor error.

public abstract class UserMixin { 
    @JsonProperty("json_id")
    String id;

    @JsonProperty("json_another_id")
    String another_id;

    @JsonProperty("code")
    int code;

    @JsonProperty("json_name")
    String name;
}

and example usage:

ObjectMapper objectMapper = new ObjectMapper();
objectMapper.addMixIn(Example.class, ExampleMixin.class);
Position usr = objectMapper.readerFor(Example.class).readValue(json);
System.out.println(json);

Exception:

om.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot find a (Map) Key deserializer for type [simple type, class com.google.protobuf.Descriptors$FieldDescriptor]

Please, help me find a good solution to this issue.

Answer 1

Classes generated by protoc compiler are not simple POJO. They contain many different methods and types we need to "filter out" to make Jackson work.

Fixed MixIn class

Indeed there is a simpler solution than custom deserialiser. You need to ignore Map<Descriptors.FieldDescriptor, Object> getAllFields() method and improve field names by adding underscore: _.

Example:

import com.celoxity.protobuf.ExampleOuterClass.Example;
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.json.JsonMapper;
import com.google.protobuf.Descriptors;
import com.google.protobuf.Timestamp;

import java.time.Instant;
import java.util.Map;

public class ProtobufApp {
    public static void main(String[] args) throws Exception {
        ObjectMapper mapper = JsonMapper.builder()
                .enable(SerializationFeature.INDENT_OUTPUT)
                .addMixIn(Example.class, ExampleMixin.class)
                .addMixIn(Timestamp.class, TimestampMixin.class)
                .build();

        String json = "{" +
                "\"json_id\":\"3\"," +
                "\"json_another_id\":\"43\"," +
                "\"code\":34," +
                "\"json_name\":\"Yeyproto\"," +
                "\"currTime\":{\"seconds\":1575909372,\"nanos\":35000000}" +
            "}";
        Example deserialised = mapper.readValue(json, Example.class);

        System.out.println(deserialised);
        Timestamp currTime = deserialised.getCurrTime();
        System.out.println(Instant.ofEpochSecond(currTime.getSeconds(), currTime.getNanos()));
    }
}

abstract class ExampleMixin extends ProtoBufIgnoredMethods {

    @JsonProperty("json_id")
    String id_;

    @JsonProperty("json_another_id")
    String anotherId_;

    @JsonProperty("code")
    int code_;

    @JsonProperty("json_name")
    String name_;

    @JsonProperty("currTime")
    Timestamp currTime_;
}

abstract class TimestampMixin extends ProtoBufIgnoredMethods {
    @JsonProperty("seconds")
    String seconds_;

    @JsonProperty("nanos")
    String nanos_;
}

abstract class ProtoBufIgnoredMethods {
    @JsonIgnore
    public abstract Map<Descriptors.FieldDescriptor, Object> getAllFields();
}

Above code prints:

id: "3"
another_id: "43"
code: 34
name: "Yeyproto"
currTime {
  seconds: 1575909372
  nanos: 35000000
}

2019-12-09T16:36:12.035Z

Custom deserialiser + com.hubspot library

In that case, the simplest solution is to write set of deserialisers and serialisers for all com.google.protobuf.* types are compiled into POJO. Luckily, there is already implemented module which handles them: jackson-datatype-protobuf.

Example usage in your case could look like below:

import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.json.JsonMapper;
import com.fasterxml.jackson.databind.module.SimpleModule;
import com.fasterxml.jackson.databind.node.ObjectNode;
import com.hubspot.jackson.datatype.protobuf.ProtobufModule;

import java.io.IOException;

public class ProtobufApp {
    public static void main(String[] args) throws Exception {
        SimpleModule pojosModule = new SimpleModule();
        pojosModule.addDeserializer(Example.class, new ExampleJsonDeserializer());

        ObjectMapper mapper = JsonMapper.builder()
                .enable(SerializationFeature.INDENT_OUTPUT)
                .addModule(new ProtobufModule())
                .addModule(pojosModule)
                .build();

        String json = "{\"json_id\":\"3\",\"json_another_id\":\"43\",\"code\":34,\"json_name\":\"Yeyproto\"}";
        Example deserialised = mapper.readValue(json, Example.class);
        System.out.println(deserialised);
    }
}

class ExampleJsonDeserializer extends JsonDeserializer<Example> {
    @Override
    public Example deserialize(JsonParser p, DeserializationContext ctxt) throws IOException {
        ObjectNode root = p.readValueAsTree();
        return Example.newBuilder()
                .setId(root.get("json_id").asText())
                .setAnotherId(root.get("json_another_id").asText())
                .setName(root.get("json_name").asText())
                .setCode(root.get("json_id").asInt())
                .build();
    }
}

Example code prints:

id: "3"
another_id: "43"
code: 3
name: "Yeyproto"

Maven dependencies:

<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.10.0</version>
</dependency>
<dependency>
    <groupId>com.google.protobuf</groupId>
    <artifactId>protobuf-java</artifactId>
    <version>3.11.0</version>
</dependency>
<dependency>
    <groupId>com.hubspot.jackson</groupId>
    <artifactId>jackson-datatype-protobuf</artifactId>
    <version>0.9.11-jackson2.9</version>
</dependency>