data is not fully transferred on LAN using TcpClient

Question

I send the serialized data over LAN network but sometimes the information is lost! Process is the following:

Sender:

1. string mydata is being serialized
2. string mydata is converted to byte[] bytes_of_mydata
3. int size_of_mydata is the length of byte[] bytes_of_mydata
4. int size_of_mydata itself is turned into byte[] bytes_size_of_mydata
5. byte[] bytes_of_mydata and byte[] bytes_size_of_mydata are sent

Receiver:

1. i first receive byte[] bytes_size_of_mydata
2. retrieve length int size_of_mydata of the second message from byte[] bytes_size_of_mydata
3. i then receive byte[] bytes_of_mydata, knowing exact length!
4. i then convert byte[] bytes_of_mydata to string mydata
5. deserialize string mydata

This approach usually works in most situations, but sometimes my data is not transmitted fully, so the string can't be deserialized.

I've debugged the received byte[] on the "receiver" and here is what happens:

I get the size of second message:

int size_of_second_message = BitConverter.ToInt32(dataByteSize, 0); // 55185

I start to receive the second message to byte array:

Byte[] dataByte = new Byte[55185];

But starting from position 5840 I start to receive 0 (nulls), so the part "5840 - 55185" are all "0":

byte[5836] = 53;
byte[5837] = 57;
byte[5838] = 54;
byte[5839] = 49;
byte[5840] = 0; // information ends to flow
byte[5841] = 0;
byte[5842] = 0;
byte[5843] = 0;
//....
byte[55185] = 0;

The example from the above is taken from an actual debugger!

So what's the problem? It's like the connection is being lost during transmission!! Why is it happening and how do I counter this problem? It doesn't happen on "every-time" basis.

And here comes the code

Send:

//text_message - my original message
//Nw - network stream
MemoryStream Fs = new MemoryStream(ASCIIEncoding.Default.GetBytes(text_message));
Byte[] buffer = Fs.ToArray(); // total 55185 bytes (as in example)
Byte[] bufferSize = BitConverter.GetBytes(Fs.Length); // 32 bytes represent size
bufferSize = GetNewByteSize(bufferSize);

Nw.Write(bufferSize, 0, bufferSize.Length); // send size
Nw.Flush();

Nw.Write(buffer, 0, buffer.Length); // send message
Nw.Flush();

Receive:

//get first(SIZE) bytes:
int ReadSize = 0; int maxSize = 32; // 32 - constant!
Byte[] dataByteSize = new Byte[maxSize];
int origsize;
using (var strm = new MemoryStream())
{
    ReadSize = Nw.Read(dataByteSize, 0, maxSize);
    strm.Write(dataByteSize, 0, ReadSize);
    strm.Seek(0, SeekOrigin.Begin);
    origsize = BitConverter.ToInt32(dataByteSize, 0); // origsize = 55185
}
Nw.Flush();

//get next(MESSAGE) bytes:
string message = ""; int thisRead = 0;
int max = Convert.ToInt32(origsize); // origsize = 55185
Byte[] dataByte = new Byte[max];

using (var strm = new MemoryStream())
{
    thisRead = Nw.Read(dataByte, 0, max);
    strm.Write(dataByte, 0, thisRead);
    strm.Seek(0, SeekOrigin.Begin);
    using (StreamReader reader = new StreamReader(strm))
    {
        message = reader.ReadToEnd();
    }
}
Nw.Flush();
// message - the message that is being transmitted partly (sometimes)! 

I didn't want to post the code but you guys usually ask "show us what you've done", so here it is!

Edit

Temporary fix is to switch to StreamWriter, reader.

Receive + send (server):

NetworkStream Nw = new NetworkStream(handlerSocket.Client);
string toreceive = "";
StreamReader reader = new StreamReader(Nw);
toreceive = reader.ReadLine();
string text_message = "to send back";
StreamWriter writer = new StreamWriter(Nw);
writer.WriteLine(text_message);
writer.Flush();
Nw.Close();

Send + receive (client):

NetworkStream Nw = new NetworkStream(handlerSocket.Client);
StreamWriter writer = new StreamWriter(Nw);
writer.WriteLine("to send");
writer.Flush();

string toreceive = new StreamReader(Nw).ReadLine();
writer.Close();
Nw.Close();

I'm looking for a solution regarding original problem, but so far everything is working due to temporary fix.

Answer 1

TCP is a stream based protocol which lets you read as much data has yet been received. Just because you send data in one step, there is no guarantee that the data will be received in the same block. You have to loop on the receiving side until you have received all the data you anticipate.

By the way I think that your protocol with an explicit length field at the start is really good, it makes up for simple client code (as simple as it gets at least).

Some time back I asked a question on built in functionality to wait until X bytes of data is available: .NET blocking socket read until X bytes are available?. The answer is unfortunately no.