TCP server receiving more data than expected

Question

I have a server-client application where a client is streaming image data to the server. I have the following structure:

Client:

private void SerializeAndSendMessage(Message msg) {
        BinaryFormatter formatter = new BinaryFormatter();
        MemoryStream stream = new MemoryStream();
        formatter.Serialize(stream, msg);
        byte[] buffer = stream.ToArray();

        if (clientSocket != null)
        {
            if (clientSocket.Connected)
            {
                clientSocket.BeginSend(buffer, 0, buffer.Length, SocketFlags.None, SendCallback, null);
            }
        }
    }

private void SendCallback(IAsyncResult ar) {
        try
        {
            clientSocket.EndSend(ar);
            Debug.WriteLine("Message sent.");
        }
        catch (Exception ex)
        {
            //
        }
    }

Server:

  private void ReceiveCallback(IAsyncResult ar)
    {
        try
        {
            int received = clientSocket.EndReceive(ar);
            Array.Resize(ref buffer, received);
            BinaryFormatter formatter = new BinaryFormatter();

            MemoryStream stream = new MemoryStream(buffer);

            object obj = null;
            stream.Position = 0;
            try
            {
                obj = formatter.Deserialize(stream);
            }
            catch (Exception ex )
            {
               //
            }

            // processing data

            Array.Resize(ref buffer, clientSocket.ReceiveBufferSize);
            clientSocket.BeginReceive(buffer, 0, buffer.Length, SocketFlags.None, ReceiveCallback, null);
        }
        catch (Exception ex)
        {
            //
        }
    }

What I expect to happen:

1. server starts accepting data from client
2. client sends data with size X
3. server receives the data with size X and starts processing it
4. client is still sending data in the meantime
5. server does not receive this data
6. server is done processing the received data and now starts receiving from client
7. goto 2

What is happening:

1. server starts accepting data from client
2. client sends data with size X
3. server receives the data with size X and starts processing it
4. client is still sending data in the meantime
5. server does not receive this data
6. server is done processing the received data and now starts receiving from client
7. client sends N-th packet with size X
8. server receives data of size M*X

This can obviously cause the buffer on the server to fill up and be unable to deserialize the packages sent. What am I missing? What can I do to achieve the workings described above?

Answer 1

TCP is a streaming protocol. If you do several send operations at the client side after eachtother, TCP is going to combine them into one segment as it is trying to fill up to the mtu.

TCP will send if the mtu is full or if a continuous 50ms timer elapses, or if the client itself has to ack a packet that it received from the server.

TCP is a very complex protocol. There is also an algorithm in there that calculates a window size. Also this window size has influence on the size of the segments that are received at the client side.

Bottom line is because TCP is streaming protocol, there is no notion of a packet that you receive through the socket. You receive an arbitrary number of bytes that you have to append to some kind of receive buffer yourself depending on what you are doing. If you desire packets, then you have to prepend the data that you send with a length field and take the length into account at the server as well. This of course complicates the code. Or if you want to keep it simple, simply use UDP. UDP does support packets and what you send will, if the packet does not get lost somewhere, be received with the same size at the receiver. But UDP is not reliable, the packet might get lost. TCP is reliable, is connection oriented, but is more complex.

Socket programming in general is not a beginners topic.