dpdk ixgbe nic multi rx queue with rss not work well

Question

I am using IXGBE Nic, using dpdk 19.11 multi rx queue with rss "ETH_RSS_TCP | ETH_RSS_IP". IXGBE support max 64 queues, and i used 4 queues. But all packages arrvied to the same queue(queue 0), it seems rss function not work well. The following is the pseudo code of my rss part. Is the rss function not taking effect due to my incorrect configuration?

static int rss_setup(const int port_id, const uint16_t nb_queues)
{
    int i;
    int ret;
    struct rte_eth_dev_info dev_info;
    struct rte_eth_rss_reta_entry64 *reta_conf = NULL;

    rte_eth_dev_info_get(port_id, &dev_info);

    if (nb_queues == 0) {
        return ERR_VAL;
    }

    reta_conf = calloc(dev_info.reta_size / RTE_RETA_GROUP_SIZE,
                       sizeof(struct rte_eth_rss_reta_entry64));
    if (!reta_conf) {
        return ERR_MEM;
    }
    for (i = 0; i < dev_info.reta_size; i++) {
        struct rte_eth_rss_reta_entry64 *one_reta_conf =
            &reta_conf[i / RTE_RETA_GROUP_SIZE];
        one_reta_conf->reta[i % RTE_RETA_GROUP_SIZE] = i % nb_queues;
    }

    for (i = 0; i < dev_info.reta_size / RTE_RETA_GROUP_SIZE; i++) {
        struct rte_eth_rss_reta_entry64 *one_reta_conf = &reta_conf[i];
        one_reta_conf->mask = 0xFFFFFFFFFFFFFFFFULL;
    }

    ret = rte_eth_dev_rss_reta_update(port_id, reta_conf, dev_info.reta_size);
    if (ret < 0) {
        printf("cannot update rss reta at port %d: %s\n",
            port_id, rte_strerror(-ret));
    }

    free(reta_conf);
    return ERR_OK;
}
main (){
    struct rte_eth_conf conf = {0}；
    conf.link_speeds = ETH_LINK_SPEED_AUTONEG;
    conf.txmode.mq_mode = ETH_MQ_TX_NONE;
    conf.rxmode.mq_mode = ETH_MQ_RX_NONE;
    
    struct rte_eth_dev_info dev_info;
    rte_eth_dev_info_get(port_id, &dev_info);
    
    int rss_enable = 0;
    uint64_t def_rss_hf = ETH_RSS_TCP | ETH_RSS_IP;
    struct rte_eth_rss_conf rss_conf = {
        NULL,
        40,
        def_rss_hf,
    };

    rss_conf.rss_hf &= dev_info->flow_type_rss_offloads;
    if (rss_conf.rss_hf) {
        rss_enable = 1;
        conf->rx_adv_conf.rss_conf = rss_conf;
        conf->rxmode.mq_mode = ETH_MQ_RX_RSS;
        rss_setup(port_id, 4); // queues cnt is 4
    }
}

Answer 1

1. Looks like the program does not invoke rte_eth_dev_configure() before invoking rss_setup(). This is not correct as rte_eth_dev_configure() "must be invoked first before any other function in the Ethernet API" (doc reference). Please re-arrange the program to first invoke rte_eth_dev_configure() with rx_adv_conf.rss_conf and ETH_MQ_RX_RSS specified, then proceed with the rest of bring-up.

2. In rss_setup(), the program invokes rte_eth_dev_rss_reta_update(). For the PMD in question, this API can only be invoked in started state (code reference). So please re-arrange the program a bit more to invoke rte_eth_dev_start() before rss_setup().

3. Please make sure that TCP source port numbers (and/or IP source addresses) are randomised at the sender side. If IP/TCP source and destination fields do not vary, it's no surprise that the packets end up in the same Rx queue.