Search code examples
vulkan

Why does VkImage require extensive format information?


When creating an image with vkCreateImage, the image's format must be specified in the VkImageCreateInfo instance.

The format contains information about the memory requirements of the image. It also contains information about the colour layout (component order), colour space (linear RGB, sRGB, unspecified), and colour conversion methods.

All this data is necessary to use an image in Vulkan. That makes complete sense. On the other hand, certain formats are compatible with each other, meaning, their memory requirements are the same, but the other parameters differ.

This can be taken advantage of, by specifying a different format for the image's view.

But then why specify the full format of an image (VkImage) at all? VK_FORMAT contains many more format specifiers than just the memory requirements. Why aren't these separate? They seem to be irrelevant for VkImage. Is there any point in considering more than the memory requirements (compatibility class) when picking a format for an image?


Solution

  • I can provide one example where that info (format, sRGB, etc) is needed to be known by the driver.

    Blitting

    This example is vkCmdBlitImage. This function copies an image onto another and, optionally, performs linear filtering.

    This function can do format conversion, if the src-dst formats are different (with some restrictions). Obviously, Vulkan can't do format conversion if it doesn't know the formats.

    Blitting can also do linear filtering. In the case of sRGB images, the pixels are first converted to linear, then interpolated, and then converted back to sRGB. That is something that can only be done if Vulkan knows that the image is sRGB.

    This is just one example, there might be more.


    Debuggers

    Specifying the format at image creation allows debuggers to display that image correctly to the user :)


    Tiling

    Images can be in linear tiling or optimal tiling. Tiling depends on the format. So when you do a layout transition on an image the driver might need to know the format. Actually some tilings are not even supported for some formats.


    Difficulty to know the size of some formats

    Usually, R8G8B8 textures use 4 bytes per pixel. But what if some GPU prefers to use only 3?

    Some compressed formats store indexing tables for blocks. Not easy to know the required size for the whole image.


    Format compatibility classes

    Not all formats are compatible between them. If you didn't provide the format, it would not be possible for the driver and validation layers to verify that the formats are compatible.