Search code examples
graphicsgpuvulkan

Where do Vulkan functions live?


I am trying to wrap my head around Vulkan and how it works at low level. I played around and read some of the APİ documentation. Also read some blogs about it. As far as I know Vulkan is just an API spec. There is no implementation in any language on Vulkan's website. SDK is there to discover the driver's .lib or .so files and then call them on your program. Driver's are implemented by vendors so I think I will not be able to read the source code. So basically is this what happens:

SDK discovers driver implementation -> I call driver's implementation -> Compiled code communicates to GPU -> Magic happens

I am trying to get my understanding of GPU pipeline a little bit better and I am not experienced in GPU field that much. When we compile the code we write it is usually compiled for platform or VM that is going to be used but where does GPU code go? How can I communicate with GPU at hardware level? Is this possible without using APIs?

According to my basic research some driver's implement it reading a certain part of RAM that is dedicated for GPU communication, some driver's directly compile the code and load and such. I am really lost on this and there is not much information on the web, assuming most of these stuff vendor specific.


Solution

  • I can only speak to the Win32 driver environment, but the principles will carry over to other operating systems.

    When you write a Vulkan application you link against the Vulkan loader (vulkan-1.dll). The loader searches the registry for some specific keys to find where each GPU manufacturer has their Vulkan Installable Client Driver (ICD). The ICD exports a handful of functions such as vk_icdGetInstanceProcAddr and vk_icdGetPhysicalDeviceProcAddr. The loader has a few functions which are publicly exported (mostly from the Vulkan 1.0 spec), but most are acquired through the functions vkGetInstanceProcAddr and vkGetDeviceProcAddr, the loader will generate a whole chain for every function, whereby it will descend through every layer that implements an intercept for that function, finally landing at the ICD. You can read more about this in the Vulkan Loader Interfacearchive spec.

    As for communication with the GPU, that is handled by a kernel mode driver, on Windows that would be the Display Miniport Driver. Almost all GPUs are PCIe devices (even integrated ones), and all the "magic" is really just Memory-Mapped IO. There's usually a block of memory which is the actual VRAM, and a control block. The control block has a lot of "register", which are really just memory addresses, that when read or written to trigger special actions within the GPU. This also shows up everywhere in embedded development, so that's usually a better avenue to research the concept. The GPU's PCIe controller handles all read and write requests, so it can trigger any action it wants when it receives such a request. A common simple register is a reset register, which when read from causes the GPU's current state to reset.

    Communication between these two on Windows is performed with GDI32.dll. It just exports a lot of functions that mostly just trigger an interrupt/syscall, that will go into dxgkrnl.sys, which will call the Display Miniport Driver.

    Direct3D does things in a similar way, except they use a callback model, and don't need to use GDI32.dll to communicate with dxgkrnl.sys.

    The whole flow is explained on Microsoft's documentation for WDDMarchive. There documentation doesn't mention Vulkan, but it's essentially the same flow as is described for OpenGL.

    None of this really matters for writing a Vulkan application. All you really need to know is that the real meat of Vulkan is implemented by each GPU vendor, and they'll handle all the nuance of getting your commands and shaders to run on the GPU. Because all of these user mode API's are implemented by the vendors specifically for communicating with their GPU, there really isn't a way to communicate with the GPU directly, that's what these API's are for.