Is code that relies on packet boundaries for a TCP stream inherently unreliable?

Question

From what I understand, TCP is a stream, and shouldn't rely on packets, or boundaries of any kind. A recv() call will block until the kernel network buffers have some data, and return that data, up to a set buffer size, which can optionally be provided to recv as an arg.

Despite this, there seems to be plenty of code out there that looks like this:

import construct as c
...

AgentMessageHeader = c.Struct(
    "HeaderLength" / c.Int32ub,
    "MessageType" / c.PaddedString(32, "ascii"),
)

AgentMessagePayload = c.Struct(
    "PayloadLength" / c.Int32ub,
    "Payload" / c.Array(c.this.PayloadLength, c.Byte),
)

connection = ...

while True:
    response = connection.recv()

    message = AgentMessageHeader.parse(response)
    payload_message = AgentMessagePayload.parse(response[message.HeaderLength:])

    print("Payload Message:", payload_message.Payload)

This code is an excerpt from a way to read data from a TCP websocket from an AWS API. Here, we appear to be trusting that connection.recv() will always receive one packet worth of data, with complete headers, and payload.

Is code like this inherently wrong? Why does it work a large majority of the time? Is it because in most cases, recv will return a full packet worth of data?

Answer 1

Yes, it's wrong.

It may work in a protocol where only one message is sent at a time in a simple request/response dialogue. You don't have to worry about recv() returning data from multiple messages that were sent back-to-back, because that never happens. And while the kernel is permitted to return less data than you request, in practice it will return as much as is available up to the buffer size.

And at the sending end, most calls to send() will result in all the data being sent in a single segment (if it fits within the MTU) or a few segments sent back-to-back. These will likely be buffered together in the receiving kernel, and returned at once in the recv() call.

You should never depend on this in production code; the examine you quoted is simplified because it's concentrating on the WebSocket features. Or the author just didn't know better.