In order to leverage the GPUs on a system, I'd like to be able to draw a block diagram and understand the connections represented by "nvidia-smi topo -m" output.
Can someone provide a system level block diagram of this? Descriptions of the connections would be great too. I'm sure this would help many people exploit their multi gpu systems.
You have two CPU sockets. Each CPU has a PCIe host bridge (PHB) leading to two PCIe switches. Each of the (two times two) PCIe switches has two GPUs connected to it, for a total of eight.