Deserialization of multiple objects with POCO and cereal in C++

Question

I'm trying to send some objects from one program to the other over the network using POCO::Net to handle the connection and cereal to handle the serialization.

All packets are inherited from single base, BasePacket.

class BasePacket {}; 
typedef std::shared_ptr<BasePacket> BasePacketPtr;

This is how I send data:

Poco::Net::StreamSocket &socket; //passed as an argument
Poco::Net::SocketOutputStream os(socket);
while (!sendQueue.empty()) {
    BasePacketPtr v;
    v = sendQueue.front();
    { //block to make sure that cereal flushes everything
     cereal::BinaryOutputArchive oa(os);
     oa << v;
    }
    sendQueue.pop();
}
os.flush();

And this is how I receive data:

Poco::Net::StreamSocket &socket; //passed as an argument
Poco::Net::SocketInputStream is(socket);
cereal::BinaryInputArchive ia(is);
BasePacketPtr v;
while(socket.available()) {
    ia >> v;
    //do sth with v...
}

The problem is that when data is send very quickly, many packets can be waiting in the socket buffer and cereal reads all the bytes, but only deserializes first packet and the rest is lost.

Is there a way to make cereal read only one packet at the time or deserialize more than one packet ? Or should I make some kind of buffer that goes between Poco's socketstream and cereal's archive that send data size so it can read one packet at the time before passing it to cereal?

Answer 1

In case anyone happens to have the same problem, here is the cause of the issue:

while(socket.available())

This only iterates once, even if there are more than one packet in the buffer (cereal archive takes all the data from socket, so it doesn't have any data available and thus it returns 0).

When new packets appear, the buffer is reset and the data is lost.

The solution is to keep reading from cereal achieve until there is no more data. If this is running in a separate thread, even while(isRunning()) { ia >> v; /.../ } is enough.