Pipeline compatibility across render-passes seems overly rigid in Vulkan?

Question

Let me just preface with an apology for the long and ranty post, but I felt I had to include some context, since I feel confused about the situation and might be misunderstanding it at some other point entirely.

I'm trying to work out how to add a Vulkan rendering backend to an application currently using OpenGL, and I'm wondering if I'm misunderstanding the parts about pipeline compatibility across different render-pass objects. The way I'm reading it, a pipeline state object can only be used with a render-pass object other than the one it was created with if the new one effectively (to simplify it) has the same number and format of attachments for the entirety of the whole render-pass.

This seems extremely inflexible to me, and I'm wondering if I'm misreading the requirement, and whether it isn't the subpass the pipeline was created with that should have compatible attachments. The reason I'm doubting my reading is that, if I'm considering the subpass graphs that it seems reasonable for my applications to have, I'd have to be doing a whole lot of massive pipeline recompiles that seem entirely unnecessary to me. To take two examples:

1. In drawing my 2D UI, which includes movable "window" widgets, I currently draw each window into its own buffer texture, and then composite the windows into the UI as a whole. It sounds very reasonable to me to have a 2D render-pass with a subpass for each window, and then a final subpass for the "compositing". Seems like something a driver might find quite productive to reorder and all that nice stuff that render-passes are supposed to be good for. However, since each window has its own buffer, just creating or destroying a window would require creating a new render-pass with more or fewer attachments (one for each window). Which would be fine, I'm sure creating a render-pass per frame if necessary wouldn't be a performance issue, but since it would change the number of attachments, according to my current reading, that would make every single pipeline object in the whole 2D pass incompatible, requiring all of them to be recompiled. I can't imagine that not causing very noticeable frame stutters just for opening or closing windows.
2. In my main 3D render-pass, I want to be able to add and remove certain post-processing effects depending on circumstances. Unless I want to have to create a render-pass with potentially global knowledge of all post-processing effects that may or may not be required at any point during the application's lifetime, that would also require creating new render-pass objects with more or fewer attachments, which again would invalidate every pipeline object in the whole 3D pass, even those in other subpasses.

What's more, this inflexibility doesn't seem to match very well with my (admittedly layman's) understanding for the reasons behind requiring render-pass compatibility. The way I understand it, the reason why pipeline objects need to be bound to a render-pass is so that the pipeline can be tightly bound to how the hardware formats the rendered data into the framebuffer, having pre-knowledge of the sample formats and such. Which makes sense to me, but what doesn't make sense to me is why that compatibility would be affected by what happens in other subpasses. A particular subpass only has a subset of the whole render-pass' attachments available to it, and as I see it, only that subset should affect how the pipeline carries out its work on the hardware anyway, no?

That's why I'm wondering if I should be reading the attachment compatibility requirements to be between subpasses rather than render-passes as a whole. Unless I want to be constantly recompiling all my pipeline objects, it seems I'm left having to simply chain together "simple" render-passes with one subpass each, not getting any of the supposed benefits of subpass graphs, and I'm wondering how many Vulkan-using applications there even exist out there that would use completely unchanging render-passes for their entire lifetimes.

Is it truly that inflexible, or am I misreading it, or am I misunderstanding something else entirely?

Answer 1

Subpasses are designed for a very specific optimization, which is allowing later subpasses to read data at an identical pixel coordinate from the earlier subpasses inside the same render pass. This can be beneficial on some GPU architectures, such as tile-based renderers found in mobile where the data can be kept on-chip. However, it's only useful for some specific use cases where there is a need for data sharing of otherwise transient data at a matching pixel coordinate between the subpasses (e.g. a deferred lighting pass reading from a geometry buffer which never leaves tile memory).

The reason for the entire render pass needing to be compatible is that all of subpasses need to know about the pixel memory layout of all the subpasses to ensure they can exchange data correctly.

It honestly doesn't sound like you need subpasses at all for what you are doing. Composition can be achieved with simple blending of draw calls, and doesn't need complex data exchange beyond that.