What are the normal methods for achiving texture mapping with raytracing?

Question

When you create a BLAS (bottom level acceleration structures) you specify any number of vertex/index buffers to be part of the structure. How does that end up interacting with the shader and get specified in the descriptor set? How should I link these structures with materials?

How is texture mapping usually done with raytracing? I saw some sort of "materials table" in Q2RTX but the documentation is non-existent and the code is sparsely commented.

Answer 1

A common approach is to use a material buffer in combination with a texture array that is addressed in the shaders where you require the texture data. You then pass the material id e.g. per-vertex or per-primitive and then use that to dynamically fetch the material, and with it the texture index. Due to the requirements for Vulkan ray tracing you can simplify this by using the VK_EXT_descriptor_indexing extension (Spec) that makes it possible to create a large and descriptor set containing all textures required to render your scene.

The relevant shader parts:

// Enable required extension
...
#extension GL_EXT_nonuniform_qualifier : enable

// Material definition
struct Material {
    int albedoTextureIndex;
    int normalTextureIndex;
    ...
};

// Bindings
layout(binding = 6, set = 0) readonly buffer Materials { Material materials[]; };
layout(binding = 7, set = 0) uniform sampler2D[] textures;

...
// Usage
void main()
{
    Primitive primitive = unpackTriangle(gl_Primitive, ...);
    Material material = materials[primitive.materialId];
    vec4 color = texture(textures[nonuniformEXT(material.albedoTextureIndex)], uv);
    ...
}

In your application you then create a buffer that stores the materials generated on the host, and bind it to the binding point of the shader.

For the textures, you pass them as an array of textures. An array texture would be an option too, but isn't as flexible due to the same size per array slice limitation. Note that it does not have a size limitation in the above example, which is made possible by VK_EXT_descriptor_indexing and is only allowed for the final binding in a descriptor set. This adds some flexibility to your setup.

As for the passing the material index that you fetch the data from: The easiest approach is to pass that information along with your vertex data, which you'll have to access/unpack in your shaders anyway:

    struct Vertex {
        vec4 pos;
        vec4 normal;
        vec2 uv;
        vec4 color;
        int32_t materialIndex;
    }