Search code examples
c++protocol-bufferspocoprotobuf-net

Protobuf ParseDelimitedFrom implementation in C++


C# Publisher is publishing continuos marketdata messages in custom protobuff format over the socket using "writeDelimitedTo" API. I have to read all messages in C++ and desearialize it. Below is my code. Since C++ don't have "parseDelimitedFrom", so have coded something like below after going through multiple suggestions in this forum.

Now my question is - Refering to the code below, If the first message size is less than 1024 then in the first iteration, i will have full stream of the 1st message and part of the stream from the 2nd message. After deserializing first message, How can i read remaining streams of the second message from socket and merge it with the stream which i read in the previous iteration ?


Solution

  • EDIT: Support for "delimited" format is now part of the official protobuf library. The post below predates it being added.

    I've written optimally-efficient versions of parseDelimitedFrom and writeDelimitedTo in C++ here (the read and write methods of Uncompressed):

    https://github.com/capnproto/capnproto/blob/06a7136708955d91f8ddc1fa3d54e620eacba13e/c%2B%2B/src/benchmark/protobuf-common.h#L101

    Feel free to copy.

    These implementations read from / write to a ZeroCopyInputStream / ZeroCopyOutputStream.(Hmm, for some reason my write is declared to use FileOutputStream, but you should be able to just change that to ZeroCopyOutputStream.)

    So, you'll need to create a ZeroCopyInputStream which reads from your StreamSocket, then pass it to my read().

    It looks like StreamSocket is a classic copying-read interface. You should therefore use CopyingInputStreamAdaptor as your ZeroCopyInputStream, wrapping an implementation of CopyingInputStream which reads from your StreamSocket.

    https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.io.zero_copy_stream_impl_lite#CopyingInputStreamAdaptor