Search code examples
javaprotocol-buffersdatamodel

Working with Protocol Buffers and internal data models


I have an existing internal data model for a Picture, as follows:

package test.model;
public class Picture {

  private int height, width;
  private Format format;

  public enum Format {
    JPEG, BMP, GIF
  }

  // Constructor, getters and setters, hashCode, equals, toString etc.
}

I now want to serialize it using protocol buffers. I've written a Picture.proto file that mirrors the fields of the Picture class and compiled the code under the test.model.protobuf package with a classname of PictureProtoBuf:

package test.model.protobuf;

option java_package = "test.model.protobuf";
option java_outer_classname = "PictureProtoBuf";

message Picture {
  enum Format {
    JPEG = 1;
    BMP = 2;
    GIF = 3;
  }
  required uint32 width = 1;
  required uint32 height = 2;
  required Format format = 3;
}

Now I am now assuming that if I have a Picture that I want to serialize and send somewhere I have to create a PictureProtoBuf object and map all the fields across, like so:

Picture p = new Picture(100, 200, Picture.JPEG);
PictureProtoBuf.Picture.Builder output = PictureProtoBuf.Picture.newBuilder();
output.setHeight(p.getHeight());
output.setWidth(p.getWidth());

I'm coming unstuck when I have an enumeration in my data model. The ugly way that I'm using right now is:

output.setFormat(PictureProtoBuf.Picture.Format.valueOf(p.getFormat().name());

However, this is prone to breakage and relies on the enumeration name being consistent between my internal data model and the protocol buffer data model (which isn't a great assumption as enumeration names within .proto files need to be unique). I can see me having to hand-craft switch statements on enumerations if the .name() call from the internal model doesn't match the protobuf-generated enumeration name.

I guess my question is whether I'm going about this the right way? Am I supposed to scrap my internal data model (test.model.Picture) in favour of the protobuf-generated one (test.model.protobuf.PictureProtoBuf)? If so, how can I implement some of the niceties that I have done in my internal data model (e.g. hashCode(), equals(Object), toString(), etc.)?


Solution

  • Although the existing answers are good, I decided to go a bit further with Marc Gravell's suggestion to look into protostuff.

    You can use the protostuff runtime module along with the dynamic ObjectSchema to create schemas at runtime for your internal data model

    My code now reduces to:

    // Do this once
    private static Schema<Picture> schema = RuntimeSchema.getSchema(Picture.class);
    private static final LinkedBuffer buffer = LinkedBuffer.allocate(DEFAULT_BUFFER_SIZE);
    
    // For each Picture you want to serialize...
    Picture p = new Picture(100, 200, Picture.JPEG);
    byte[] result = ProtobufIOUtil.toByteArray(p, schema, buffer);
    buffer.clear();
    return result;
    

    This is a great improvement over the Google protobuf library (see my question) when you have lots and lots of attributes in your internal data model. There is also no speed penalty that I can detect (with my use cases, anyway!)