Search code examples
clinuxperformance-testingmicrobenchmarkrdma

How to build perftest and run latency tests on RDMA

I'm trying to build the perftest C library to run some latency tests over the RDMA protocol.

My steps:

  • I downloaded the library from their GitHub and unzipped it on the box on which I want to run the server for the tests

  • I executed:

    ./autogen

    ./configure

    make clean && make V=1


  • I tried to run a test

    ./ib_send_lat --duration=30 -H

..but I get the following error:

Port number 1 state is Down
Couldn't set the link layer
Couldn't get context for the device

What am I doing wrong?

Running other tests that use the RDMA protocol (eg. the ones in Accelio) the system shows no problems.

Also, I runned a few diagnostic tools and everything seems configured correctly already:

  • Verified that RDMA kernel part is loaded (ibuverbs present) enter image description here

  • Verified that RDMA devices are available enter image description here

  • Verified that RDMA devices can be accessed (the second one says PORT_ACTIVE) enter image description here

  • Verified that traffic over RDMA is working sending messages from a client to a server using rping enter image description here

Any suggestions? (I opened an issue on their GitHub but I got no reply yet)

Thanks

Solution

  • I suspect the problem is the perftest tools are defaulting to using the device that's in the PORT_DOWN state.

    Try specifying the specific device to use with the '-d' option. eg:

    ib_send_lat -d mlx5_0 ...

    (This will probably need to be done on both sides of the link.)