Very very high speed serial communication over serial port in C# with best perfomance and without any data lost

Question

I have a data losing problem over C# serial port in an sample app with very very high speed data receiving (up to 36000 byte per second in 2000 datapacks with 18 byte) from a micro controller board.

My baudrate is 1.000.000 and I use DataReceived event. And I have a simulator to generate data for main application test. I paired two COM port (COM1 -> COM2) with "Virtual Serial Port Driver" software.

After the datapacks have been received, they are saved to a *.CSV file with some data getting lost.

My first and important problem is the data loss.

My second problem is CPU usage performance in this case (up to 50% or 60%).

This is my simulator serialport config and data generator code:

SerialPort serialPort = new SerialPort("COM1", 1000000, Parity.None, 8, StopBits.One);
serialPort.Handshake = Handshake.None;
serialPort.Open();
serialPort.DataReceived += new SerialDataReceivedEventHandler(sp_DataReceived);

byte[] packet = new byte[] { 83, 84, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 69, 78 };
ulong packetCount = 0;
while (packetCount < 600000);
{
    byte[] timeBytes = BitConverter.GetBytes(packetCount);
    packet[2] = timeBytes[0]; 
    packet[3] = timeBytes[1]; 
    packet[4] = timeBytes[2]; 
    packet[5] = timeBytes[3];
    Write(packet);
    packetCount++;
}

In this code, packet has a header {83, 84} and footer {69, 78}

This is serial port config in my main application:

SerialPort _serialPort = new SerialPort("COM2", 1000000, Parity.None, 8, StopBits.One);
_serialPort.Handshake = Handshake.None;
_serialPort.DataReceived += new SerialDataReceivedEventHandler(sp_DataReceived);
_serialPort.ReadBufferSize = 100;

This is the dataReceived event in my main application:

private void sp_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
    int count = _serialPort.BytesToRead;
    byte[] receivedTemp = new byte[count];
    _serialPort.Read(receivedTemp, 0, count);
    _totalReceived.AddRange(receivedTemp);
    findPacket();
}

This is the findPacket function to find a packet using its header and footer:

int findPacket(int index = 0, bool headerFounded = false)
{
    while (index + 1 < _totalReceived.Count)
    {
        if (_totalReceived[index] == 83 && _totalReceived[index + 1] == 84)
        {
            int headerIndex = index;
            int packetLen = 14; 
            int footerIndex = findPacket(index + 2 + packetLen, true);
            if (footerIndex > -1 && (packetLen == 0 || (footerIndex == headerIndex + packetLen + 2)))
            {
                List<byte> data = totalReceived.GetRange(headerIndex + 2, footerIndex - (headerIndex + 2));
                totalReceived.RemoveRange(headerIndex, footerIndex + 2 - headerIndex);
                if (headerIndex > 0 && !headerFounded)
                    totalReceived.RemoveRange(0, headerIndex);

                calcPacket(data);
                continue;
            }
            else if (footerIndex != -1)
            { 
                 index++; 
                 continue; 
            }
            break;
        }

        if (headerFounded && totalReceived[index] == 69 && totalReceived[index + 1] == 78)
            return index;

        index++;
    }

    return -1;
}

Problem is: findPacket is very very busy when simulator data generator is started, and all of other async functions (like main UI and CSV writer and online chart drawer and ...) in my main app are not working and locked.

Answer 1

You can do this without using the recursive function. Recursive functions increase the resources used by the system. In this case, the number of calls may be excessive and cause a time error you can do this:

  async void findPacket3()
    {
        byte[] header = new byte[] { 83, 84 };
        byte[] footer = new byte[] { 69, 78 };
        int maxLength = header.Length + footer.Length;

        int index = 0;
        int headerIndex = -1;
        int footerIndex = -1;
        while (!_canceltoken.IsCancellationRequested)
        {
            await Task.Delay(5);
            while (index < totalRecieved.Count)
            {
                if (index + (header.Length - 1) < totalRecieved.Count && totalRecieved[index] == header[0] && totalRecieved[index + 1] == header[1])
                {
                    headerIndex = index;
                    int packetLen = 0; // findPacketLen(index);
                    index += header.Length + packetLen;
                    continue;
                }
                if (headerIndex > -1 && index + (footer.Length - 1) < totalRecieved.Count && totalRecieved[index] == footer[0] && totalRecieved[index + 1] == footer[1])
                {
                    footerIndex = index;
                    List<byte> data = totalRecieved.GetRange(headerIndex + header.Length, footerIndex - (headerIndex + header.Length));
                    calcPacket(data);
                    headerIndex = -1;
                }
                index++;
            }
            if (footerIndex > -1)
            {
                totalRecieved.RemoveRange(0, footerIndex + footer.Length);
                footerIndex = -1;
                index = 0;
                headerIndex = -1;
            }
        }

    }

Also, in order to avoid data loss, it is better to set

_serialPort.ReadBufferSize = 4096;