Search code examples
dpdk

How do you calculate DPDK mempool requirements?

How do you figure out what settings need to be used to correctly configure a DPDK mempool for your application?

Specifically using rte_pktmbuf_pool_create():

  • n, the number of elements in the mbuf pool
  • cache_size
  • priv_size
  • data_room_size

EAL arguments:

  • n number of memory channels
  • r number of memory ranks
  • m amount of memory to preallocate at startup
  • in-memory no shared data structures
  • IOVA mode
  • huge-worker-stack

My setup:

  • 2 x Intel Xeon Gold 6348 CPU @ 2.6 Ghz
    • 28 cores per socket
    • Max 3.5 Ghz
    • Hyperthreading disabled
    • Ubuntu 22.04.1 LTS
    • Kernel 5.15.0-53-generic
    • Cores set to performance governor
    • 4 x Sabrent 2TB Rocket 4 Plus in RAID0 Config
    • 128 GB DDR4 Memory
    • 10 1GB HugePages (Can change to what is required)
  • 1 x Mellanox ConnectX-5 100gbe NIC
    • 31:00.0 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]
    • Firmware-version: 16.35.1012
  • UDP Source:
    • 100 gbe NIC
    • 9000 MTU Packets
    • ipv4-udp packets

Will be receiving 10GB/s UDP packets over a 100gbe link. Plan is to strip the headers and write the payload to a file. Right now trying to get it working for 2GB/s to a single queue.

Reviewed the DPDK Programmers guide: https://doc.dpdk.org/guides/prog_guide/mempool_lib.html Also searched online but the resources seem limited. Would appreciate any help or a push in the right direction.

Solution

  • based on the updates from comments the question can be summarized as

    what are the correct settings to be used for DPDK mbuf|mempool which needs to handle 9000B UDP payload for processing 10Gbps packets on 100Gbps MLX CX-5 NIC with single or multiple queues

    Let me summarize my suggestions below for this unique case

    [for 100Gbps]

    1. as per MLX DPDK performance report for test case 4, for packet size 1518 we get theoretical and practical Million Packets per sec as 8.13
    2. Hence for 9000B payload this will be, 9000B/1518B=6 is around 8.13/6 = 1.355 MPps
    3. With MLX CX-5 1 queue achieve a mx of 36Mpps - so with 1 queue and JUMBo enabled, you should get the 9000B into a single queue

    note: 10Gbps it will be 0.1355Mpps

    Settings for MBUF or mempool:

    1. if your application logic requires 0.1 seconds to process the payload, I recommend you to use 3 * max expected packets. So roughly 10000 packets
    2. Each payload has total size of 10000B (data_room_size) as single contiguous buffer.
    3. priv_size is wholly dependant upon your logic to store metadata

    Note: in case multiple queue, I always configure for worst case scenario, that is I assume there will be elephant flow which can fall onto specific queue. So if with 1 queue you have created 10000 elements, for multiple queues I use 2.5 * 10000