Why does libvirt create multiple PCIs?

Question

I ran libvirt in the parameters of the qemu-system-x86_64 command and found that it created multiple PCIs:

/usr/bin/qemu-system-x86_64 \
    ...
    -device '{"driver":"pcie-root-port","port":16,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x2"}' \
    -device '{"driver":"pcie-root-port","port":17,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x2.0x1"}' \
    -device '{"driver":"pcie-root-port","port":18,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x2.0x2"}' \
    -device '{"driver":"pcie-root-port","port":19,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x2.0x3"}' \
    -device '{"driver":"pcie-root-port","port":20,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x2.0x4"}' \
    -device '{"driver":"pcie-root-port","port":21,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x2.0x5"}' \
    -device '{"driver":"pcie-root-port","port":22,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x2.0x6"}' \
    -device '{"driver":"pcie-root-port","port":23,"chassis":8,"id":"pci.8","bus":"pcie.0","addr":"0x2.0x7"}' \
    -device '{"driver":"pcie-root-port","port":24,"chassis":9,"id":"pci.9","bus":"pcie.0","multifunction":true,"addr":"0x3"}' \
    -device '{"driver":"pcie-root-port","port":25,"chassis":10,"id":"pci.10","bus":"pcie.0","addr":"0x3.0x1"}' \
    -device '{"driver":"pcie-root-port","port":26,"chassis":11,"id":"pci.11","bus":"pcie.0","addr":"0x3.0x2"}' \
    -device '{"driver":"pcie-root-port","port":27,"chassis":12,"id":"pci.12","bus":"pcie.0","addr":"0x3.0x3"}' \
    -device '{"driver":"pcie-root-port","port":28,"chassis":13,"id":"pci.13","bus":"pcie.0","addr":"0x3.0x4"}' \
    -device '{"driver":"pcie-root-port","port":29,"chassis":14,"id":"pci.14","bus":"pcie.0","addr":"0x3.0x5"}' \
    -device '{"driver":"pcie-root-port","port":30,"chassis":15,"id":"pci.15","bus":"pcie.0","addr":"0x3.0x6"}' \
    -device '{"driver":"pcie-pci-bridge","id":"pci.16","bus":"pci.6","addr":"0x0"}' \
    ...

It seems that qemu only creates one PCI by default. Is there any benefit to creating multiple PCIs for virtual IO devices?

Answer 1

With a PCI-Express topology, every individual device (disk, nic, video, sound, etc) needs to be attached to a dedicated pcie-root-port. You cannot hotplug/unplug the pcie-root-port device itself, but if you already have a pcie-root-port present you can hotplug/unplug a device into it. Thus to allow you to hotplug disks/nics/etc at runtime, it is desirable to pre-create extra pcie-root-port devices. Libvirt will only pre-create one spare, but apps using libvirt can ask for as many extras as they like. Seeing your config with 30 ports, I'm guessing you might have been using OpenStack since I know that will pre-create many extra pcie-root-port to allow for lots of hotplug options.