Search code examples
infinibandmellanoxofed

What is the difference between OFED, MLNX OFED and the inbox driver

I'm setting up Infiniband networks, and I do not fully get the difference between the different software stacks.

How are they related to each other and why should I prefer one of these amongst the others?

Solution

    1. MLNX_OFED has embedded performance tweaks that can only be utilized with it. In most scenarios, with a well tuned MLNX_OFED you will be able to see benchmarking advantage.

    2. MLNX_OFED has features that inbox does not have. You can compare the features with the release notes of each driver depending on the OS:

    Release notes for Inbox driver: http://www.mellanox.com/page/inbox_drivers

    Release notes for MLNX_OFED drivers: http://www.mellanox.com/page/software_overview_ib

    1. Mellanox cannot guarantee the latest driver related code (functionality, bug fixes and performance optimizations) is included in each OS and kernel. The drivers are constantly being updated, and submitting and qualifying patches to upstream kernel revolves around a heavy process before any commit actually gets accepted. A commit first gets accepted to upstream kernel and then cherry picked to a specific distro kernel. With Mellanox OFED you can rest assured you are up to date regardless if your use case is IB, Ethernet or just an application using libibverbs.

    2. Most of the packages in MLNX_OFED differ than the ones that come with inbox driver. e.g. IPoIB, libmlx5, and others.

    3. Depending on your needs and application you are using, with Mellanox drivers you don't "have" to install the entire package, For Ethernet you can just use the Mellanox ethernet driver, with an extension of libibverbs if you use RDMA based applications. You can also use Mellanox OFED to specifically cherry-pick packages you need, or alternatively create your own repository using included RPMs. The point is that you get presented with a number of options, and you get to choose. The recommended version is whatever proves to work with your environment in the most stable way.

    OFED is an OpenFabrics distribution of the RDMA/Advanced Networks code base. The majority of the code is pulled from github.com/linux-rdma and git.kernel.org. That code is then sometimes enhanced with new modifications being tested by vendors, and/or back ported patches.

    The inbox driver is the version that holds the most generic code. In case your deployment is based on MLNX hardware - I would go with the MLNX_OFED driver as it provides the latest and most enhanced features for the MLNX products, and is the most flexible in terms of installation etc.

    For your reference you can check the the following articles for different Mellanox OFED solutions:

    Mellanox Linux Driver Modules Relationship (MLNX_OFED) https://community.mellanox.com/docs/DOC-2081

    Linux Driver Solutions: https://community.mellanox.com/docs/DOC-2287

    In addition: Support for Inbox drivers should be provided by your OEM or directly from the OS/Distro vendor.