I've created a server in C# and am trying to efficiently handle data that comes in from clients. I currently have this code for checking for client packets on a client thread:
if ( myStream.DataAvailable == true && myStream.CanRead ) {
int mySize = myStream.ReadByte();
myBuffer = new byte[ mySize ];
await myStream.ReadAsync( myBuffer, 0, mySize );
ushort myPeek = 0;
Net_Messages.Net_Read( myStream, myID, myBuffer, myPeek, myThreading );
}
So my question is, is it efficient to store the data from the stream into a buffer for each packet received from each client? Should I have a fixed predefined buffer instead of creating a new one for each packet retrieval?
How should I write data as well? Create a fixed pre-defined buffer for each client, store data on that buffer and send the buffer?
My problem with pre-defined buffers is allocating extra memory that is not needed by creating an unnecessarily sized buffer when I can create a new buffer for each packet at the correct size.
NOTE: Net_Read is a simple pass through function that passes the buffer and extra info as needed for processing the received packet.
You certainly shouldn't usually allocate new buffer for each read, if that is what you mean. But beyond that, it depends a lot on context. For example, if incoming data is rare (web-sockets, for example), you might want to use a tiny buffer (even just a single byte) for the async read - and then use a larger buffer (perhaps from a pool) when data is available. Or you might want to use socket-polling instead! For busier connections it is not uncommon to end up using two buffers per connection - one fixed-size for reading, and one (perhaps a MemoryStream) as a back-buffer while you check for entire frames (which could come via multiple reads). Sometimes you can combine them (for example, SE.Redis uses the read buffer as the back-buffer, resizing it if needed, and copying data backwards in the buffer if we can consume some frames but have data left over).
There is no such thing as "efficient handling" - it depends entirely on context. What is efficient for one usage pattern could be highly inefficient for another. The number of concurrent connections and the frequency of reads are key factors.