Low-latency communication between threads in the same process

Question

Console application has 3 threads: Main, T1, T2. The goal is to 'signal' both T1, T2 (and let them do some work) from the Main thread in the lowest latency as possible (μs)

NOTE:

• please ignore Jitter, GC etc. (I can handle that)
• ElapsedLogger.WriteLine call cost is below 50ns (nano sec)

Have a look at the code below:

sample 1

class Program
{
    private static string msg = string.Empty;
    private static readonly CountdownEvent Countdown = new CountdownEvent(1);

    static void Main(string[] args)
    {
        while (true)
        {
            Countdown.Reset(1);
            var t1 = new Thread(Dowork) { Priority = ThreadPriority.Highest };
            var t2 = new Thread(Dowork) { Priority = ThreadPriority.Highest };
            t1.Start();
            t2.Start();

            Console.WriteLine("Type message and press [enter] to start");
            msg = Console.ReadLine();

            ElapsedLogger.WriteLine("Kick off!");
            Countdown.Signal();

            Thread.Sleep(250);
            ElapsedLogger.FlushToConsole();
        }
    }
    private static void Dowork()
    {
        string t = Thread.CurrentThread.ManagedThreadId.ToString();
        ElapsedLogger.WriteLine("{0} - Waiting...", t);

        Countdown.Wait();

        ElapsedLogger.WriteLine("{0} - Message received: {1}", t, msg);
    }
}

Output:

Type message and press [enter] to start
test3
20141028 12:03:24.230647|5 - Waiting...
20141028 12:03:24.230851|6 - Waiting...
20141028 12:03:30.640351|Kick off!
20141028 12:03:30.640392|5 - Message received: test3
20141028 12:03:30.640394|6 - Message received: test3

Type message and press [enter] to start
test4
20141028 12:03:30.891853|7 - Waiting...
20141028 12:03:30.892072|8 - Waiting...
20141028 12:03:42.024499|Kick off!
20141028 12:03:42.024538|7 - Message received: test4
20141028 12:03:42.024551|8 - Message received: test4

In the above code 'latency' is around 40-50μs. CountdownEvent signaling call is very cheap (less than 50ns) but T1,T2 threads are suspended and it takes time to wake them up.

sample 2

class Program
{
    private static string _msg = string.Empty;
    private static bool _signal = false;

    static void Main(string[] args)
    {
        while (true)
        {
            _signal = false;
            var t1 = new Thread(Dowork) {Priority = ThreadPriority.Highest};
            var t2 = new Thread(Dowork) {Priority = ThreadPriority.Highest};
            t1.Start();
            t2.Start();

            Console.WriteLine("Type message and press [enter] to start");
            _msg = Console.ReadLine();

            ElapsedLogger.WriteLine("Kick off!");
            _signal = true;

            Thread.Sleep(250);
            ElapsedLogger.FlushToConsole();
        }
    }
    private static void Dowork()
    {
        string t = Thread.CurrentThread.ManagedThreadId.ToString();
        ElapsedLogger.WriteLine("{0} - Waiting...", t);

        while (!_signal) { Thread.SpinWait(10); }

        ElapsedLogger.WriteLine("{0} - Message received: {1}", t, _msg);
    }
}

Output:

Type message and press [enter] to start
testMsg
20141028 11:56:57.829870|5 - Waiting...
20141028 11:56:57.830121|6 - Waiting...
20141028 11:57:05.456075|Kick off!
20141028 11:57:05.456081|6 - Message received: testMsg
20141028 11:57:05.456081|5 - Message received: testMsg

Type message and press [enter] to start
testMsg2
20141028 11:57:05.707528|7 - Waiting...
20141028 11:57:05.707754|8 - Waiting...
20141028 11:57:57.535549|Kick off!
20141028 11:57:57.535576|7 - Message received: testMsg2
20141028 11:57:57.535576|8 - Message received: testMsg2

This time 'latency' is around 6-7μs. (but high CPU) This is because T1,T2 threads are forced to be active (they doing nothing just burn CPU time)

In 'real' application I cannot spin CPU like that (I have far to many active threads and it would make it worse/slower or even kill the server).

Is it anything I can use instead to drop latency to something around 10-15 μs? I guess with Producer/Consumer pattern it won't make is quicker than using CountdownEvent. Wait/Pulse is also more expensive than CountdownEvent.

Is what I got in sample 1 the best I can achieve?

Any suggestions?

I'll try raw sockets as well when I have a time.

Answer 1

There's not a whole lot that can be done, since the other thread has to be scheduled by the OS.

Increasing the priority of the waiting thread is the only thing likely to make much difference, and you've already done that. You could go even higher.

If you really need the lowest possible latency for activation of another task, you should turn it into a function that can be called directly from the triggering thread.