Search code examples
c#c++.netasynchronouslmdb

Await in C# for an event in C++ - as a task or a wait handle


I program mostly in .NET and I love its async/concurrency primitives such as Tasks, ResetEvents, etc. Today for the first time I made meaningful changes to a C++ program and understood how the whole build process works (I updated LigthningDB.NET project to 0.9.14). But I still lack C++ knowledge.

One new feature that I want to add to the LMDB project (for my own needs) is a notification system (similar to Redis):

  • I want a cursor returning an awaitable object that will signal on every data change in its table.
  • I want to get some data together with the signal (a pointer to a data structure), e.g. key or key + value.
  • This object will be disposed together with the cursor and is a part of the cursor, but it signals on changes in a DB for which the cursor has a reference.
  • This must work cross platform some day (or some year). I am OK with any dirty Windows-specific hacks if they do the job.

Typical use case is a writer and N readers waiting for new messages. This will allow IPC and fast persistence 2-in-1. LMDB supports concurrent reading from different processes (while Esent and LevelDB do not).

System.Threading primitives are available from C++ and I understand how to use WaitHandle for a blocking call. Is there a way to make this async? There is a great series of articles about async synchronization primitives, but they use TaskCompletionSource and that works only inside .NET. Is it possible to make a similar solution for native interop?

One solution could be named pipes or socket translating changes to one listener/dispatcher (Redis or Rhino.Queues style), but performance will suffer: writes will have to allocate, copy and push data and data will have to travel - much worse than passing a pointer to a data structure that is already in memory.

Another option is to move the listening cursor to the key and send a signal. After the signal a C# listener will know that the cursor has values at the updated key. This kinf of solves data transfer part, but with WaitHandles it is blocking - in my use case blocking is worse than the sockets [allocation/copy/delay] combination.

Are there better options?

Additional questions:

  1. Am I reinventing a bicycle here?
  2. Could you please point to open source libraries where .NET program awaits (non-blocking) for a C/C++ signal (if they exist)?
  3. Should I use LMDB for that workflow? Windows in my priority and I have had very bad time trying to make LevelDB work at all (stopped trying). Are there better alternatives for everything LMDB does + signalling?

Update

I have found in the documentation this method:

public static Task WaitOneAsync(this WaitHandle waitHandle)
{
    if (waitHandle == null) throw new ArgumentNullException("waitHandle");

    var tcs = new TaskCompletionSource<bool>();
    var rwh = ThreadPool.RegisterWaitForSingleObject(waitHandle, 
        delegate { tcs.TrySetResult(true); }, null, -1, true);
    var t = tcs.Task;
    t.ContinueWith(_ => rwh.Unregister(null));
    return t;
}

The doc for ThreadPool.RegisterWaitForSingleObject says that:

The wait operation is performed by a thread from the thread pool. The delegate is executed by a worker thread when the object's state becomes signaled or the time-out interval elapses. ... The wait thread uses the Win32 WaitForMultipleObjects function to monitor registered wait operations.

Am I correct that these are two different threads, and the first wait thread is the only one that will block if there are no signals?


Solution

  • RWFSO uses special thread pool threads to wait for multiple handles. There's a built-in limit of 63 handles per thread, so this is not as efficient as IOCPs. I wouldn't recommend a handle-based solution even though MREs can be used to solve this (MREs can be used to solve pretty much everything...).

    C++ does not have a notion of "events". The traditional approach is to take a callback function pointer (which, combined with Boost.Bind and Boost.Function is not nearly as painful these days). A more modern approach would be Boost.Signals2.