Search code examples
macoscocoametal

Transparent MTKView not blending properly with windows behind it

I have a trivial macOS app that has an MTKView--it is just an MTKView inside a window. The fragment shader always does:

return float4(1, 0, 0, 0.01)

That's red with 1% alpha.

The CAMetalLayer belonging to the MTKView is non-opaque, as is the window:

_mtkView.layer.opaque = NO;
_mtkView.window.opaque = NO;
_mtkView.window.backgroundColor = [NSColor clearColor];
_mtkView.window.hasShadow = NO;

When I put this window over another app's window, I would expect the other app's pixels to show through unchanged. Instead, I see that black pixels turn red and white pixels are mostly unchanged.

In the screenshot below, the triangle has color float4(1,0,0,0.01) and there's a half-black half-white image in another window showing through it:

Screenshot showing the problem

I would have expected black ever-so-slightly tinged with red, not bright red, in the left half of the triangle.

If my fragment shader is changed to do this:

return float4(1, 0, 0, 0.001);

Then it is completely transparent: pixels from other windows show through completely unchanged. This makes me think I'm hitting a not-well-tested code path in the window server, but I'm hoping there's something I can do to make it work.

What I tried

  • Changing the appearance of the NSThemeFrame (had no effect)
  • Turning on presentsWithTransaction (no effect)
  • The GPU debugger confirms the drawable has the expected value
Solution

  • Apple wrote:

    The CoreAnimation compositor requires all of its inputs be premultipled by alpha. See these references for more information: https://en.wikipedia.org/wiki/Alpha_compositing https://blogs.msdn.microsoft.com/shawnhar/2009/11/06/premultiplied-alpha/

    When RGB are premultipled by alpha, then any RGB component that is > the alpha component is undefined through the hardware’s blending.

    Modifying your fragment shader to return (0.1, 0, 0, 0.1), which is “0.1 alpha, full red” produces the expected results.

    Indeed, this fixed my issue. The relevant commit is https://github.com/gnachman/iTerm2/commit/70a93885b1f67bda843ba546aca0eca67b750088 where I modified the program to draw to an offscreen texture and then copy it to the drawable while multiplying the red, green, and blue components by alpha.

    For some reason that I cannot understand, my test program also required conversion to linear color space.