Coordinating descriptor sets and layouts in Vulkan

I've been doing research into Vulkan and raytracing through NVidia's vk_raytrace example and I've run into confusion about how coordination between pipeline layouts, GLSL shader "layouts", and descriptor sets happen.

Primary question:

I really want to create a single "layout" file which can be shared across all shader files regardless of pipeline. Is this possible if the pipeline layouts match only the used descriptor sets within the shader? Would the unused layouts be optimized out when compiling the shader to SPIR-V?

Other questions:

You don't specify the actual "descriptor set" number when creating a pipeline layout, simply some ordering. In the NVidia example they create a "shared" descriptor set between two pipelines, in the main pipeline this is set 1 and in the other its set 0. What is going on here - can set numbers be totally arbitrary?
I've been looking for examples for multiple push-constants for pipelines - how does this work - and are there any official examples using GLSL?

Relevant code sections pertaining to the primary question:

Creation of the Post pipeline where it uses its own descriptor set

void RenderOutput::createPostPipeline(const VkRenderPass& renderPass)
{


 // This descriptor is passed to the RTX pipeline
  // Ray tracing will write to the binding 1, but the fragment shader will be using binding 0, so it can use a sampler too.
  bind.addBinding({OutputBindings::eSampler, VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER, 1, VK_SHADER_STAGE_FRAGMENT_BIT});
  bind.addBinding({OutputBindings::eStore, VK_DESCRIPTOR_TYPE_STORAGE_IMAGE, 1,
                   VK_SHADER_STAGE_COMPUTE_BIT | VK_SHADER_STAGE_RAYGEN_BIT_KHR});
  m_postDescSetLayout = bind.createLayout(m_device);
  m_postDescPool      = bind.createPool(m_device);
  m_postDescSet       = nvvk::allocateDescriptorSet(m_device, m_postDescPool, m_postDescSetLayout);
  //...
  pipelineLayoutCreateInfo.pSetLayouts = 
    &m_postDescSetLayout;
  //...
  vkCreatePipelineLayout(m_device, &pipelineLayoutCreateInfo, nullptr, &m_postPipelineLayout);

Creation of the Raygen pipeline which uses the shared descriptor set m_offscreen.getDescSet:

void SampleExample::createRender(RndMethod method)
{
  //...

  m_pRender[m_rndMethod]->create(
      m_size, {m_accelStruct.getDescLayout(),
               m_offscreen.getDescLayout(),  // This is the shared desclayout
               m_scene.getDescLayout(), 
               m_descSetLayout}, &m_scene);
}

Here are the corresponding shader layouts - for the Post pipeline the only relevant file is "post.frag"

//...
layout(location = 0) in vec2 uvCoords;
layout(location = 0) out vec4 fragColor;

layout(set = 0, binding = 0) uniform sampler2D inImage;

layout(push_constant) uniform _Tonemapper
{
  Tonemapper tm;
};
//...

The rest of the layouts get included by most? of the raygen shaders layouts - "layouts.glsl":

// clang-format off
layout(set = 0, binding = 0)                    uniform accelerationStructureEXT topLevelAS;

// Shared layout
layout(set = 1,   binding = 1)                  uniform image2D         resultImage;
//...

Solution

To make sure we're on the same page, let's start with some terminology.

When I say "pipeline layout", what I mean is a VkPipelineLayout object and the various components used to build it (descriptor set layouts, etc). There are also descriptor definitions in a shader, which are... the definitions of descriptor resources potentially used by that shader. These are not "pipeline layouts" (ignore the fact that the keyword layout is used to declare the descriptor and set indices; that's just convenient syntax). Lastly, there are the actual VkDescriptorSet objects you create and bind at runtime.

You don't specify the actual "descriptor set" number when creating a pipeline layout

Yes you do. Set X refers to the X'th index of the array VkPipelineLayoutCreateInfo::pSetLayouts. Set 0 is the descriptor set pSetLayouts[0].

If the NVIDIA example uses the same descriptor set layout with different set indices in different pipeline layouts, then binding a pipeline that uses the descriptor set layout in a different set index will disturb the bound descriptor sets. So it must also rebind the descriptor set to the appropriate set index.

That is, the code in question is not trying to maintain binding compatibility between the pipeline layouts; it just wants to be able to use the same descriptor set.

It is fine for a shader module to not define all of the descriptors specified in a pipeline layout. Indeed, it is fine if some descriptors in the pipeline layout is defined by none of the shader modules when building a pipeline. The descriptor set layout is what matters; so long as a shader module does not contradict the pipeline layout, a VkDescriptorSet built against a particular layout will work with a VkPipeline object using a VkPipelineLayout that includes that descriptor set layout.