Search code examples
c++jsonstreamnlohmann-json

Parsing JSON data from TCP stream

I am using nlohmann's json library for parsing json data from a TCP stream. I am not quite sure how to handle partial json reads from local socket. Suppose that in the first read() I get:

{ 
    "MessageType": "CancelOrder",
    "Account":11111, 
    "CustomerNo":11111,
    "Side":"A",
    "DestinationMarket":"DUMB_MARKET",
    "Symbol":"DUMB_SYMBOL",
    "PositionEffect":"D",
    "Limi

and in the following read() from socket, I get:

    tPrice":0,
    "Quantity":1,
    "OrderType":"DUMB_TYPE",
    "StopPrice":0,
    "TimeInForce":"01.06.1999",
    "ExpireDate":0,
    "OrderID": "DUMB_ID",
    "IsStopOrder":"DUMB_STOP",
    "CorrelationId": 456
}

Partial reads cannot be parsed by the library since they are not valid. Does the library offer a solution to this? Or should I implement a solution myself?

What should be the best practice here?

Solution

  • You've gotten some good answers in the comments. I'm going to assemble some and add one more choice.

    If you have control over both ends of the communications, then some people feel you should change the communications in one of two ways:

    • Send the length of text first
    • Or use a smarter messaging system over the socket

    Either of these would solve your problem for you.

    I'll offer two more possible choices.

    • Send an "end of data" indicator -- something that won't appear in the JSON. For instance, a null-byte. Break before the EOD character.
    • Try successively parsing data until it parses successfully.

    The second one is kind of ugly. You'd parse { and get an exception. Then you'd parse {" for an exception, over and over until finally you have complete JSON. I bet it's slow, but it might work, and it doesn't depend on changing the data stream in any way.

    Personally, I'd consider in order:

    1. Use a proper messaging protocol
    2. Use an End of Data indicator
    3. Send the length
    4. The hack of parsing and catching the exceptions until it parses

    I think any of these would work. The last one is the only one that doesn't force you to change both ends of your data stream.