Search code examples
javajsonprotocol-buffers

Java: Protobuf byte to Json string to Pojo fast

I am receiving messages in protobuf format. I need to convert it to json format fast as all my business logic is written to handle json based POJO objects.

byte[] request = ..; // msg received

// convert to intermediate POJO
AdxOpenRtb.BidRequest bidRequestProto = AdxOpenRtb.BidRequest.parseFrom(request, reg);

// convert intermediate POJO to json string.
// THIS STEP IS VERY SLOW
Printer printer = JsonFormat.printer().printingEnumsAsInts().omittingInsignificantWhitespace();
String jsonBody = printer.print(bidRequestProto);

// convert json string to final POJO format
BidRequest bidRequest = super.parse(jsonBody.getBytes());

Proto object to json conversion step is very slow. Is there any faster approach for it?

can i reuse printer object? is it thread-safe?

Note: This POJO class (AdxOpenRtb.BidRequest & BidRequest) is very complex having many hierarchy and fields but contains similar data with slightly different fields name and data types.

Solution

  • I end up using MapStruct as suggested by some of you (@M.Deinum).

    new code:

    byte[] request = ..; // msg received

// convert to intermediate POJO
AdxOpenRtb.BidRequest bidRequestProto = AdxOpenRtb.BidRequest.parseFrom(request, reg);

// direct conversion from protobuf Pojo to my custom Pojo
BidRequest bidRequest = BidRequestMapper.INSTANCE.adxOpenRtbToBidRequest(bidRequestProto);

    Code snippet of BidRequestMapper:

    @Mapper(
    collectionMappingStrategy = CollectionMappingStrategy.ADDER_PREFERRED, nullValueCheckStrategy = NullValueCheckStrategy.ALWAYS,
    unmappedSourcePolicy = ReportingPolicy.WARN, unmappedTargetPolicy = ReportingPolicy.WARN)
@DecoratedWith(BidRequestMapperDecorator.class)
public abstract class BidRequestMapper {

  public static final BidRequestMapper INSTANCE = Mappers.getMapper(BidRequestMapper.class);

  @Mapping(source = "impList", target = "imp")
  @Mapping(target = "impOverride", ignore = true)
  @Mapping(target = "ext", ignore = true)
  public abstract BidRequest adxOpenRtbToBidRequest(AdxOpenRtb.BidRequest adxOpenRtb);
...
...
}

// manage proto extensions
abstract class BidRequestMapperDecorator extends BidRequestMapper {
  private final BidRequestMapper delegate;

  BidRequestMapperDecorator(BidRequestMapper delegate) {
    this.delegate = delegate;
  }

  @Override
  public BidRequest adxOpenRtbToBidRequest(AdxOpenRtb.BidRequest bidRequestProto) {
    // Covert protobuf msg to basic bid request object
    BidRequest bidRequest = delegate.adxOpenRtbToBidRequest(bidRequestProto);
...
...
  }
}

    The new approach is 20-50x faster in my local test environment.

    It's worth mentioning that MapStruct is an annotation processor which makes it much faster than other similar libraries which use reflection and it also has very good support for customization.