After enough playing with asynchronous socket programming I noticed that the server was receiving chunked payloads (ie: more than one complete payload sitting in the same buffer). So I came up with the following:
if (bytes_to_read > 0)
{
while (bytes_to_read > 0)
// Get payload size as int.
// Get payload in byte format.
// Do something with payload.
// Decrease the amount of bytes to read.
}
// Wait for more data.
}
And then I noticed packet fragmentation (ie: what I thought were complete payloads chunked together wasn't always so) which changed the previous code to something like:
if (bytes_to_read > 0)
{
while (bytes_to_read > 0)
{
// Get payload size as int.
// Check if the payload size is less than or equal to the amount of bytes left to read.
if (payload_size <= bytes_to_read)
{
// Get payload in byte format.
// Do something with payload.
// Decrease the amount of bytes to read.
}
else
{
// We received a fragmented payload.
break;
}
}
if (bytes_to_read == 0)
{
// Wait for more data.
}
else if (bytes_to_read > 0)
{
// Wait for more data where we left off. ***
}
else
{
// Something awful happened.
}
}
*** I don't even know how to go about this and would like to see code for it. I had an idea that it involved copying the in-completed payload to the beginning of the buffer and then picking up from there.
The pseudo code I included is based on the Begin* End* method I am using (I'm aware that I should be using the *Async set of methods found here -> http://msdn.microsoft.com/en-us/library/system.net.sockets.socketasynceventargs.aspx <- but I think my overall question still applies).
I am seeking the answers to 2 questions--namely:
EDIT: I'm using raw sockets.
Thanks in advance for all your help.
EDIT: John Saunders and Greg Hewgill have brought up the point of treating data as a stream but that does not provide me with a concrete example of how to deal with the last chunked payload sometimes being fragmented.
EDIT: I have read Jon Skeet's answer here which is basically along the same lines as the other answers I have seen but it doesn't help me much as I already get what I have to do but not how to do it.
EDIT: To elaborate on what I mean by fragmentation, consider the following the receive buffers:
EDIT: I found this and this which lead me here. Vadym Stetsiak has cleared nearly everything up (his was one of the answers I was looking for).
When you have to do it yourself, it can be done like so (reference here):
///
/// Server state holds current state of the client socket
///
class AsyncServerState
{
public byte[] Buffer = new byte[512]; //buffer for network i/o
public int DataSize = 0; //data size to be received by the server
//flag that indicates whether prefix was received
public bool DataSizeReceived = false;
public MemoryStream Data = new MemoryStream(); //place where data is stored
public SocketAsyncEventArgs ReadEventArgs = new SocketAsyncEventArgs();
public Socket Client;
}
///
/// Implements server receive logic
///
private void ProcessReceive(SocketAsyncEventArgs e)
{
//single message can be received using several receive operation
AsyncServerState state = e.UserToken as AsyncServerState;
if (e.BytesTransferred <= 0 || e.SocketError != SocketError.Success)
{
CloseConnection(e);
}
int dataRead = e.BytesTransferred;
int dataOffset = 0;
int restOfData = 0;
while (dataRead > 0)
{
if (!state.DataSizeReceived)
{
//there is already some data in the buffer
if (state.Data.Length > 0)
{
restOfData = PrefixSize - (int)state.Data.Length;
state.Data.Write(state.Buffer, dataOffset, restOfData);
dataRead -= restOfData;
dataOffset += restOfData;
}
else if (dataRead >= PrefixSize)
{ //store whole data size prefix
state.Data.Write(state.Buffer, dataOffset, PrefixSize);
dataRead -= PrefixSize;
dataOffset += PrefixSize;
}
else
{ // store only part of the size prefix
state.Data.Write(state.Buffer, dataOffset, dataRead);
dataOffset += dataRead;
dataRead = 0;
}
if (state.Data.Length == PrefixSize)
{ //we received data size prefix
state.DataSize = BitConverter.ToInt32(state.Data.GetBuffer(), 0);
state.DataSizeReceived = true;
state.Data.Position = 0;
state.Data.SetLength(0);
}
else
{ //we received just part of the headers information
//issue another read
if (!state.Client.ReceiveAsync(state.ReadEventArgs))
ProcessReceive(state.ReadEventArgs);
return;
}
}
//at this point we know the size of the pending data
if ((state.Data.Length + dataRead) >= state.DataSize)
{ //we have all the data for this message
restOfData = state.DataSize - (int)state.Data.Length;
state.Data.Write(state.Buffer, dataOffset, restOfData);
Console.WriteLine("Data message received. Size: {0}",
state.DataSize);
dataOffset += restOfData;
dataRead -= restOfData;
state.Data.SetLength(0);
state.Data.Position = 0;
state.DataSizeReceived = false;
state.DataSize = 0;
if (dataRead == 0)
{
if (!state.Client.ReceiveAsync(state.ReadEventArgs))
ProcessReceive(state.ReadEventArgs);
return;
}
else
continue;
}
else
{ //there is still data pending, store what we've
//received and issue another BeginReceive
state.Data.Write(state.Buffer, dataOffset, dataRead);
if (!state.Client.ReceiveAsync(state.ReadEventArgs))
ProcessReceive(state.ReadEventArgs);
dataRead = 0;
}
}
}
I did not do it exactly this way myself but it helped.