Search code examples
rdma

How to get the server to write to memory region of client device over RDMA


As the title suggest I'm trying to send data over to the client (via RDMA) instead of the client sending to the server. All of the examples I can find on the topic is client to the server. Is there any resources/references i can refer to?


Solution

  • When using the libibverbs library for RDMA verbs, in most cases there is no notion of a client and a server. One establishes a connection between two queue pairs using out-of-band channel (TCP or RDMA CM) and then the configures the QPs with the connection information that was exchanged. Once the connection has been established both sides can use RDMA one-sided operation if the QP configuration allows it.

    An example of such application is perftest, which exchanges connection information with TCP (or RDMA CM) and then measures performance of RDMA one-sided and two-sided operations.

    With librdmacm and rsockets, there is a concept of a server (listening ID or socket) and a client. Still, after a connection is established, both sides are allowed to use RDMA. You can find examples for both in the rdma-core repository. For example, rping uses RDMA CM and invokes RDMA reads and writes from the server to the client, and riostream uses rsockets and invokes RDMA writes from both the client and the server.