Search code examples
cudagpumpipci-emulti-gpu

Understanding "nvidia-smi topo -m" output


In order to leverage the GPUs on a system, I'd like to be able to draw a block diagram and understand the connections represented by "nvidia-smi topo -m" output.

Here is an example output: enter image description here

enter image description here

Can someone provide a system level block diagram of this? Descriptions of the connections would be great too. I'm sure this would help many people exploit their multi gpu systems.


Solution

  • You have two CPU sockets. Each CPU has a PCIe host bridge (PHB) leading to two PCIe switches. Each of the (two times two) PCIe switches has two GPUs connected to it, for a total of eight. enter image description here