MPI.NET: run multiple sequential sessions

Question

I've been playing around with MPI.NET a little and I have come across an issue I can't seem to work around.

When creating tests for my code which runs MPI.NET, I have two unit tests which are run by MSTest sequentially, each of which creates a communicator and disposes it.
When doing this you end up in a situation where the communicator is both initialized and disposed in the second test (static variables are the cause here).

I'm not looking to solve this from a test perspective (running the tests differently won't solve the problem). I'm simply showing you that case in order to easily demonstrate the problem.

I'm hoping there is a sensible way to init MPI.NET twice, or somehow recycling it?

I did get in touch with the author of the project and he suggested I post here where someone more familiar with MPI itself might be able to help.

public static void RunMPIAction(Action<Intracommunicator> action)
{
    string[] args = null;
    using (var env = new Environment(ref args))
    {
        action(Communicator.world);
    }
}

MPI.Environment.RunMPIAction(comm =>
{
    this.Run(comm, logger, parameters);
}

MPI.Environment.RunMPIAction(comm => { int rank = comm.Rank; }, false);

Answer 1

MPI is not designed to be used like this. Creating new Environment calls MPI_Init_thread, which states:

This function must be called by one thread only. That thread will be known as the “Main Thread” and must be the same thread to call MPI_Finalize.

Attempting to call it from multiple threads or from one thread multiple times will fail.

Disposing Environment calls MPI_Finalize, which states:

The MPI_Finalize function cleans up all state related to MPI. Once it is called, no other MPI functions may be called, including MPI_Init and MPI_Init_thread.

From that it should be clear that you have to have only one Environment per process, and you cannot recreate it once disposed.

So you have to refactor your code to take this into account. If you have some kind of reusable library - don't initialize MPI there (just check if it is initialized already and throw if not) and not finalize it there - only the "top-level" user of that library should do that. If you don't have reference to Environment instance that was created but you need to finalize environment, you can do it like this:

if (!MPI.Environment.Finalized)
    Unsafe.MPI_Finalize();

That's ok to do because Environment is semantically a singleton and could have been a static class, but because of convenient using statement was made non-static (with only instance member being Dispose function).