Suppose I've got SR-IOV passthrough enabled on the host with 2 Virtual Functions, I'm running two QEMU/KVM VM's with libvirt, each connected to a VF respectively, is there any way to see the VF counters on the host (such as rx/tx pkts)?
I've tried to use ethtool -S to see stats but I can only see the global counters of the physical function.
I found an SR-IOV counters plugin for OpenStack Ceilometer but it's a Mellanox plugin and uses a proprietary drivers on the Guest VM's.
Any help would be appreciated.
I have tested this solution on Ubuntu, running on a SRIOV enabled compute in openstack installation. Steps to get counters for individual VF of a PF. The physical NIC (25G) is running on INTEL i40 driver for XXV710 device.
#lspci |grep -i ethernet
3b:00.1 Ethernet controller: Intel Corporation Ethernet Controller XXV710 for 25GbE SFP28 (rev 02)
.....
# ethtool -i eth2
driver: i40e
version: 2.7.29
firmware-version: 7.20 0x80009f02 1.2829.0
expansion-rom-version:
bus-info: 0000:3b:00.1
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
Step 1-
find for sriov dir in /sys/devices:
# find /sys/devices/ -type d -name sriov
/sys/devices/pci0000:3a/0000:3a:00.0/0000:3b:00.1/sriov
Step 2 : Go to this dir and do ls -lrt. You will see the VF numbers. These numbers are same as seen in output of "ip link show ' command. Do the corelation.
# find /sys/devices/ -type d -name sriov |xargs ls -lrt
/sys/devices/pci0000:3a/0000:3a:00.0/0000:3b:00.1/sriov:
total 0
drwxr-xr-x 3 root root 0 Jan 9 21:40 7
drwxr-xr-x 3 root root 0 Jan 9 21:40 15
drwxr-xr-x 3 root root 0 Jan 9 21:40 5
drwxr-xr-x 3 root root 0 Jan 9 21:40 13
drwxr-xr-x 3 root root 0 Jan 9 21:40 3
drwxr-xr-x 3 root root 0 Jan 9 21:40 11
drwxr-xr-x 3 root root 0 Jan 9 21:40 1
drwxr-xr-x 3 root root 0 Jan 9 21:40 8
drwxr-xr-x 3 root root 0 Jan 9 21:40 6
drwxr-xr-x 3 root root 0 Jan 9 21:40 14
drwxr-xr-x 3 root root 0 Jan 9 21:40 4
drwxr-xr-x 3 root root 0 Jan 9 21:40 12
drwxr-xr-x 3 root root 0 Jan 9 21:40 2
drwxr-xr-x 3 root root 0 Jan 9 21:40 10
drwxr-xr-x 3 root root 0 Jan 9 21:40 0
drwxr-xr-x 3 root root 0 Jan 9 21:40 9
output of ip link show
## ip link show eth2
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq portid 3cfdfea85df9 state UP mode DEFAULT group default qlen 1000
link/ether 3c:fd:fe:a8:5d:f9 brd ff:ff:ff:ff:ff:ff
vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
vf 1 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
vf 2 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
vf 3 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
vf 4 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
vf 5 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
vf 6 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
vf 7 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
vf 8 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
vf 9 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
vf 10 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
vf 11 MAC fa:16:3e:19:8c:05, vlan 1929, spoof checking off, link-state auto, trust on
vf 12 MAC fa:16:3e:43:88:4f, vlan 1927, spoof checking off, link-state auto, trust on
vf 13 MAC fa:16:3e:4f:b3:d0, vlan 1925, spoof checking off, link-state auto, trust on
vf 14 MAC fa:16:3e:56:b6:4a, vlan 1923, spoof checking off, link-state auto, trust on
vf 15 MAC 02:f1:f1:04:01:03, vlan 1938, spoof checking off, link-state auto, trust on
you can do 1 to 1 mapping and identify which is the vf you are interested in. Assume its vf 15.
Step 3- go to dir /.
# cd 15/stats
Step 4 - do ls and cat all files , except tx_spoofed
# ls -lrt
total 0
-r--r--r-- 1 root root 4096 Jan 9 21:40 tx_errors
-r--r--r-- 1 root root 4096 Jan 9 21:40 rx_packets
-r--r--r-- 1 root root 4096 Jan 9 21:40 tx_dropped
-r--r--r-- 1 root root 4096 Jan 9 21:40 tx_spoofed
-r--r--r-- 1 root root 4096 Jan 9 21:40 tx_packets
-r--r--r-- 1 root root 4096 Jan 9 21:40 rx_bytes
-r--r--r-- 1 root root 4096 Jan 9 21:40 rx_dropped
-r--r--r-- 1 root root 4096 Jan 9 21:40 tx_bytes
# tail *
==> rx_bytes <==
108786154191
==> rx_dropped <==
0
==> rx_packets <==
975217621
==> tx_bytes <==
433665695588
==> tx_dropped <==
0
==> tx_errors <==
0
==> tx_packets <==
590997020
# tail *
==> rx_bytes <==
108786397727
==> rx_dropped <==
0
==> rx_packets <==
975220967
==> tx_bytes <==
433665771018
==> tx_dropped <==
0
==> tx_errors <==
0
==> tx_packets <==
590998121
These are your counters.
BR// Sanjeev Tewari