efficient one-sided communication

Question

I am looking for an efficient one-sided communication library that can be used to implement RDMA efficiently.

Currently, I've looked at MPI-2 implementations such as MPICH2 and also MVAPICH2 (InfiniBand Architecture based implementation of MPI-2 that uses RDMA).

MVAPICH2 has a lot of functionality that I am looking for, unfortunately, I can't use that because I have no access to InfiniBand Architecture in my University currently.

So what I really want is, are there any other libraries (not necessary MPI implementations variants) that can be considered efficient in sense of reducing overhead between communicating processes (example: avoiding handshake in rendezvous protocol, thus offload overhead from the target process).

I would also appreciate any advice that helps me do something different instead of finding a cooked suitable library.

thanks.

Answer 1

First, RDMA and RMA are not the same thing. RDMA needs some sort of specialized hardware (such as InfiniBand) in order to access application buffers directly. If you do one-sided RMA operations via TCP for example, you are still going through the OS kernel and making several data copies in the process.

You could write your application using one-sided MPI operations, test it with an MPI library that runs on Ethernet, and when the time comes and you get access to faster hardware just replace the MPI library with something that supports RDMA.

Alternatively, look at some lower-level libraries such as ARMCI and GASNet. But nothing can do RDMA with no specialized hardware.

Is there a specific application you have in mind with these requirements?