Search code examples
dpdk

DPDK packet lost and disorder


I did a simple test program using DPDK: program1 in computer1 is to send packet, and program2 in computer2 is to receive packet. computer1 and computer2 are directly connected, no switch.

In program 1, I use a packId to indicate the sequence of packet id.

while(true){
pkt = rte_pktmbuf_alloc(mbuf_pool); 
uint8_t* pchar = rte_pktmbuf_mod(pkt, uint8_t*); 
//set mac address and packet length. (pkt 0 to pkt 13). 
//use from byte 14 to store uint_64 packId; 
uint64_t* pPackId = (uint64_t*)(pchar+14); 
*pPackId = packId; 
packId++; 
//put 1024 bytes data inside packet. 
uint16_t sent = rte_eth_tx_burst(0, 0, &pkt, 1); 
while(sent!=1)
{
   sent = rte_eth_tx_burst(0, 0, &pkt, 1); 
}
}

In receiver, i define long RX ring: nb_rxd=3072:

rte_eth_dev_adjust_nb_rx_tx_desc(0, &nb_rxd, &nb_txd); 
rte_eth_rx_queue_setup(0, 0, nb_rxd, rte_eth_dev_socket_id(0), NULL, mbuf_pool);

There is a for loop to receive packets, and check packet sequence id.

for(;;)
{
   strcut rte_mbuf *bufs[32]; 
   const uint16_t nb_rx = rte_eth_rx_burst(0, 0, bus, 32);
   if(unlikely(nb_rx==0))
      continue; 
   int m = 0; 
   for (m=0; m<nb_rx;m++)
   {
       uint8_t* pchar = rte_pktmbuf_mtod(buf[m], uint8_t*); 
       uint64_t* pPackId = pchar+14; 
       uint64_t packid = *pPackId; 
       if(expectedPackid!=packid){
           printf... 
           expectedPackid = packid+1; 
        }
       else expectedPackid++; 
   } 
} 

Based on program2, I see a lot of packet lost and disorder. The received packet is put inside the ring buffer. Should it receive in order, and I also find there is packet lost, but my program1's sending speed is only around 1gbps.


Solution

  • rte_eth_stats_get() is very useful for troubleshooting. From the rte_eth_stats, I found ipackets is correct, q_ipackets[0] is correct, and imissed is 0, ierrors is 0, rx_nombuf is 0, q_errors[0] is 0. So it should be codes in program2 has problem. After check codes, it is because some memory management in program2.