How to migrate a legacy Spring Boot API, which uses JSON arrays as payload, to Protobuf?

Question

Let's assume I have the following proto definition:

message Course {
  int32 id = 1;
  string course_name = 2;
}

And the following legacy Controller (Spring Boot) that needs to be backwards compatible:

@RestController
public class CourseController {
  @Autowired
  CourseRepository courseRepo;

  @RequestMapping("/courses/{id}")
  Course customer(@PathVariable Integer id) {
    return courseRepo.getCourse(id);
  }

  @PostMapping("/courses")
  Course post(@RequestBody Course course) {
    courseRepo.add(course);
    return course;
  }

  @PostMapping("/courses-bulk")
  Collection<Course> bulk(@RequestBody List<Course> courses) {
    for (Course c : courses) {
      courseRepo.add(c);
    }
    return courseRepo.getAll();
  }
}

In my Application class, I am using

@Bean
ProtobufHttpMessageConverter protobufHttpMessageConverter() {
  return new ProtobufHttpMessageConverter();
}

Instead of using ProtobufHttpMessageConverter, it appears that Spring MVC is falling back to Jackson, which trying to interpret the type as a POJO:

Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException:
Cannot find a (Map) Key deserializer for type [simple type, class com.google.protobuf.Descriptors$FieldDescriptor]

Questions:

1. Is it at all possible to deserialize JSON arrays with ProtobufHttpMessageConverter?
2. If not, how can I make Jackson work with Protobuf POJOs, so that I can use Jackson as a fallback if ProtobufHttpMessageConverter can't deserialize a JSON array payload?

Answer 1

This is how I solved my problem.

The first thing to understand is that with Protobuf, there appears to be no concept of a collection (or array) as the root of a message. The "root" of a message can only be message struct, which in turn can then contain a collection of objects. So I created the following message type:

message Courses {
  repeated Course course = 1;
}

Protobuf forces a convention here, but sending collections as a field has the benefit of allowing backward and forward compatibility, as opposed having a collection as the root element of a message.

1. Is it at all possible to deserialize JSON arrays with ProtobufHttpMessageConverter?

Yes. I was able to use Jackson's JsonNode type as a generic type to handle the array case. I split up the following mapping:

@PostMapping("/courses-bulk")
Collection<Course> bulk(@RequestBody List<Course> courses)

... into two mappings.

• Any request with application/json media type will hit the first mapping (both legacy and non-legacy case)

  • The generic JsonNode type is used instead of List
  • In the case the root JSON element is an array (legacy case), deserialize every array element by manually calling protobuf's JsonParser on each JSON element.
  • If the root JSON element is not an array, we treat the entire message as a Courses protobuf message, but encoded as JSON

• Any request with application/x-protobuf media type will hit the second mapping be automatically be deserialized from byte format into the Courses protoc-compiled Java class.

Here is the working code:

@RestController
public class CourseController {
  ...

  @PostMapping(value = "/courses-bulk", consumes = "application/json", produces = "application/json")
  Object bulk(@RequestBody JsonNode rootNode) throws InvalidProtocolBufferException, JsonProcessingException {
    Courses.Builder coursesBuilder = Courses.newBuilder();
    JsonFormat.Parser parser = JsonFormat.parser().ignoringUnknownFields();

    // JSON array is legacy case
    if (rootNode.isArray()) {
      // manually parse each JSON array element using Protobuf
      // and create Courses wrapper object
      for (JsonNode item : rootNode) {
        String itemJsonStr = item.toString();
        Course.Builder courseBuilder = Course.newBuilder();
        parser.merge(itemJsonStr, courseBuilder);
        coursesBuilder.addCourse(courseBuilder.build());
      }

      // call other bulk mapping
      Courses result = bulk(coursesBuilder.build());

      // unwrap Courses result object and convert it back to a JSON array
      ObjectMapper mapper = new ObjectMapper();
      ArrayNode arrayNode = mapper.createArrayNode();
      for (Course c : result.getCourseList()) {
        String jsonStr = JsonFormat.printer().print(c);
        JsonNode node = mapper.readTree(jsonStr);
        arrayNode.add(node);
      }
      return arrayNode;
    } else {
      // if payload is not an array, we can assume that it is a regular
      // protobuf payload, encoded as JSON
      String rootJsonStr = rootNode.toString();
      parser.merge(rootJsonStr, coursesBuilder);
      return bulk(coursesBuilder.build());
    }
  }

  @PostMapping(value = "/courses-bulk", consumes = "application/x-protobuf", produces = "application/x-protobuf")
  Courses bulk(@RequestBody Courses courses) {
    for (Course c : courses.getCourseList()) {
      courseRepo.add(c);
    }
    return Courses.newBuilder().addAllCourse(courseRepo.getAll()).build();
  }
}