Search code examples
network-programmingserializationnettyavro

How to split serialized byte stream (e.g avro, protobuf) in network?


Recently, I want to write an application using netty.The format of message I want to send is serialized object stream using Avro or Protobuf. There exists one question for me, that is while one side receive the byte streams from the other side, how could I split the byte stream,or how could I know if such stream terminated,and ready for the next serialized objects? I get some tips that is using special characters between different object byte stream,but doesn't avro or protobuf will generate such characters while serialize objects?


Solution

  • I think you most likely want to "prefix" each serialised object with the number of bytes it is serialized too. This will allow you to ensure you only read the correct number of bytes per Object and so do the right thing when de-serialize it.

    Netty itself contains for example the LengthFieldBasedFrameDecoder which will allow you to "slice" out the bytes for an object. And the LengthFieldPrepender which allows you to prefix each of them when do the encoding.