Search code examples
tapdpdk

DPDK l3fwd-acl application is unable to send or receive using net_tap PMD using namespaces

I compiled dpdk 19.11.11 version to test l3fwd-acl. After starting it in a VirtualBox instance with 2 CPUs and 2 ports I have used 2 TAP interfaces to simulate 2 network interfaces for putting traffic over one and getting that back over another. I have used the following command (and get the following output):

sam@sam-VirtualBox:~/prac/dpdk-stable-19.11.11/examples/l3fwd-acl/build/app$ sudo ./l3fwd-acl -l 1 -n 1 --vdev=net_tap0 --vdev=net_tap1 -- -p 0x3 -P --config="(0,0,1),(1,0,1)" --rule_ipv4="./ipv4_rules.db" --rule_ipv6="./ipv6_rules.db"
[sudo] password for sam:
EAL: Detected 2 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: PCI device 0000:00:03.0 on NUMA socket -1
EAL:   probe driver: 8086:100e net_e1000_em
EAL: PCI device 0000:00:08.0 on NUMA socket -1
EAL:   probe driver: 8086:100f net_e1000_em
EAL: PCI device 0000:00:09.0 on NUMA socket -1
EAL:   probe driver: 8086:100f net_e1000_em
EAL: PCI device 0000:00:0a.0 on NUMA socket -1
EAL:   probe driver: 8086:100f net_e1000_em
Promiscuous mode selected
ACL option are:
rule_ipv4: ./ipv4_rules.db
rule_ipv6: ./ipv6_rules.db
scalar: 0
L3FWDACL: IPv4 Route entries 1:
        1:0.0.0.0/0 0.0.0.0/0 0 : 65535 0 : 65535 0x0/0x0 0xffffffff-0x1ffffffe-0x2
L3FWDACL: IPv4 ACL entries 1:
        1:0.0.0.0/0 0.0.0.0/0 0 : 65535 10101 : 10101 0x6/0xfe 0xffffffff-0x1fffffff-0xf0000000
L3FWDACL: IPv6 Route entries 1:
        1:0000:0000:0000:0000:0000:0000:0000:0000/0 0000:0000:0000:0000:0000:0000:0000:0000/0 0 : 65535 0 : 65535 0x0/0x0 0xffffffff-0x1fffffff-0x1
L3FWDACL: IPv6 ACL entries 0:
acl context <l3fwd-acl-ipv40>@0x1004d8040
  socket_id=0
  alg=3
  max_rules=100000
  rule_size=96
  num_rules=2
  num_categories=1
  num_tries=1
acl context <l3fwd-acl-ipv60>@0x100fb0440
  socket_id=0
  alg=3
  max_rules=100000
  rule_size=192
  num_rules=1
  num_categories=1
  num_tries=1
Initializing port 0 ... Creating queues: nb_rxq=1 nb_txq=1... Port 0 modified RSS hash function based on hardware support,requested:0x3bffc configured:0
 Address:08:00:27:E8:C1:2B, Allocated mbuf pool on socket 0
txq=1,0,0
Initializing port 1 ... Creating queues: nb_rxq=1 nb_txq=1... Port 1 modified RSS hash function based on hardware support,requested:0x3bffc configured:0
 Address:08:00:27:1D:45:A5, txq=1,0,0

Skipping disabled port 2

Skipping disabled port 3

Initializing rx queues on lcore 1 ... rxq=0,0,0 rxq=1,0,0

Checking link status..............................done
Port0 Link Up. Speed 1000 Mbps full-duplex
Port1 Link Up. Speed 1000 Mbps full-duplex
L3FWD: entering main loop on lcore 1
L3FWD:  -- lcoreid=1 portid=0 rxqueueid=0
L3FWD:  -- lcoreid=1 portid=1 rxqueueid=0

After that I assigned an IP address for dtap0 and dtap1 following the commands shown in the accepted solution at:No traffic emerges in DPDK RX/TX Callbacks Sample Application

I tried ping from one tap interface to another using the IP assigned but cannot get any response. What could be the problem?

[EDIT 2] Using the above steps we are able to ping from tap0 namespace to namespace tap1 but we observed packet loss during that communication.

The rules files are:
1) ipv4_rules.db
+++
R0.0.0.0/0  12.12.12.2/32 0 : 65535 0 : 65535 0/0xfe 1
R0.0.0.0/0  12.12.12.1/32 0 : 65535 0 : 65535 0/0xfe 0
+++
2) ipv6_rules.db
+++
R0:0:0:0:0:0:0:0/0 0:0:0:0:0:0:0:0/0 0 : 65535 0 : 65535 0x00/0x00 0
 +++
Apart from ping with packet loss, not able to exchange traffic using "netcat" or "socat" though.  Commands used for netcat/socat were:
a) Listener side:
sudo ip netns exec tap1 nc -l 1567
OR
sudo ip netns exec tap1 socat TCP4-LISTEN:1567,fork EXEC:cat
b) Sender side:
sudo ip netns exec tap0 nc 12.12.12.2 1567
Solution

  • L3fwd-acl works with IP packets only, for all non IP packets it drops. Hence with TAP PMD I had to follow the changes as

    1. Disable any NIC PCIe device using --no-pci
    2. Use new commands as sudo ./build/l3fwd-acl --file-prefix=mytest -l 2-4 --vdev=net_tap0 --vdev=net_tap1 --no-pci -- -P -p 3 --config="(0,0,4),(1,0,3)" --rule_ipv4=ipv4-sample.db --rule-ipv6=ipv6-sample.db --eth-dest=0,<mac address of DPDK TAP0> --eth-dest=1,<mac address of DPDK TAP1>
    3. edit ip-sample.db file with the entries to be assigned for dtap0 and dtap1.
    4. refer section 24.1.3 to identify the format and for default drop rule
    5. start the application and use the following netns commands.
    sudo ip netns add tap0
sudo ip netns add tap1
ip netns
ip netns exec tap0 ifconfig -a
ip netns exec tap1 ifconfig -a
sudo ip link set dtap0 netns tap0
sudo ip link set dtap1 netns tap1
ip netns exec tap0 ifconfig -a
ip netns exec tap1 ifconfig -a
sudo ip netns exec tap0 ifconfig dtap0 12.12.12.1 up
sudo ip netns exec tap1 ifconfig dtap1 12.12.12.2 up

    If we perform sudo ip netns exec tap0 ping 12.12.12.2 the ARP packets will be dropped since it is not IP packet. To prevent this from happening, add static ARP entries for dtap0 and dtap1 within namespace tap0 and tap1. This will make the kernel to sent out ICMP packets will be then routed by L3FWD ACL program.

    Note: one can test the same using DPDK L2FWD with --file-prefix=test --no-pci --vdev=net_tap0 --vdev=net_tap1 -l 12-13 -- -p 3 -T 1 --no-mac-updating

    [EDIT-1] Question: how to assign static MAC for TAP PMD? Answer: please refer the TAP PMD link for using option mac='desired mac address' to achieve the same.

    [EDIT-2] with new linux kernel multiple rx-tx queues are created for TAP PMD> With DPDK example l3fwd-acl based on total lcores RX-TX are created. hence if we do not read from all queues packets are missed.

    To work around this problem only use 1 lcore.

    CMD: sudo ./build/l3fwd-acl --file-prefix=mytest -l 2 --no-pci --vdev=net_tap0,mac=10:20:30:40:50:66 --vdev=net_tap1,mac=10:20:30:40:50:60 -- -P -p 3 --config="(0,0,2),(1,0,2)" --rule_ipv4=ipv4-sample.db --rule_ipv6=ipv6-sample.db --eth-dest=0,10:20:30:40:50:66 --eth-dest=1,10:20:30:40:50:60

    Proc-info: sudo ./dpdk-proc-info --no-pci --file-prefix=mytest -- -p 3 --stats | grep -v ": 0"

    enter image description here