NdisProt takes a long time to write Ethernet packets

Question

I'm developing a virtualized application in C# that needs network access. I'm using the ndisprot sample driver to read and write Ethernet layer-2 packets. I have everything working fine, except that the WriteFile operations take a HUGE amount of time. Usually between 300-800ms but sometimes it takes several seconds for a write to complete.

Here is my C# function that writes an Ethernet packet to the driver. I'm using a stopwatch to measure how long it takes.

private Stopwatch sw = new Stopwatch();

/// <summary>
/// Writes a layer-2 Ethernet packet to the adapter
/// </summary>
/// <param name="packet">The packet data</param>
/// <returns>true if the write was successful</returns>
public bool WriteEnetPacket(byte[] packet)
{
   int bytesSent = 0;
   bool packetSent = false;

   // Copy the packet data to raw buffer
   IntPtr packetPtr = Marshal.AllocHGlobal(packet.Length);
   Marshal.Copy(packet, 0, packetPtr, packet.Length);

   sw.Restart();
   packetSent = WriteFile(this.handle, 
      packetPtr, 
      packet.Length,
      out bytesSent, 
      0);

   sw.Stop();
   Console.WriteLine("WriteFile completed in {0}ms", sw.Elapsed.Milliseconds);

   // Free the memory
   Marshal.FreeHGlobal(packetPtr);

   // check to see if packet was sent
   if (!packetSent)
   {
      var err = UnsafeMethods.GetLastError();
      var errStr = UnsafeMethods.GetLastErrorString();

      Console.WriteLine("ERROR: Packet not sent: 0 bytes written.");
      Console.WriteLine("Reason: " + errStr);
      return false;
   }

   // otherwise the packet was sent
   Console.WriteLine("Packet sent: " + bytesSent.ToString() + "bytes written");
   return true;
}

When I run my application, my output looks like this. I'm on a wired connection.

Packet sent: 42bytes written
WriteFile completed in 160ms
Packet sent: 42bytes written
WriteFile completed in 458ms
Packet sent: 74bytes written
WriteFile completed in 364ms
Packet sent: 74bytes written
WriteFile completed in 51ms
Packet sent: 86bytes written
WriteFile completed in 221ms
Packet sent: 74bytes written
WriteFile completed in 271ms
Packet sent: 74bytes written
WriteFile completed in 1ms
Packet sent: 74bytes written
WriteFile completed in 292ms

If I ping my application, the response times almost exactly match the time it takes for WriteFile to complete so I know the hangup is in the driver somewhere. Is there anything I can do in the sample driver to speed this up? What am I doing wrong?

Edit: Today I tried using my USB-Ethernet adapter instead of the built in adapter and performance is even worse! It's taking 15-30 SECONDS to finish writing. This is most bizarre as reading is near instantaneous.

Edit2: My driver has two threads - a read thread and a write thread. It seems that if I disable my reader thread that the writes complete instantly. Why would reading from a file affect writing to it?

Answer 1

Why are reads are slowing writes?

Your usermode code probably opened the handle in synchronous mode. While in synchronous mode, there can only be 1 call serviced at a time per handle, even if 2 different threads are working on the same handle. More here.

The fix is either:

• open N handles to get N simultaneous operations; or
• open the handle in OVERLAPPED mode. If you have the luxury of coding in C#, you would use FileOptions.Asynchronous.

Why does it take 200ms to send a single packet?

The NDISPROT sample is designed to be super simple. It has writes block until the NIC has completed the packet. That's why writes are hundreds of milliseconds. Furthermore, because the handle is opened in synchronous mode, that means you can't send the second packet until the first is completed.

That's way too slow for any serious throughput; you need to introduce asynchrony somewhere. See more discussion here: Raw ethernet broadcasting