trying to figure out how I can render a shadow on an invisible plane, so the background of my sceneView shows through.
THREE.js has a ShadowMaterial which does exactly this - only the shadow is rendered.
Current thinking is to make a custom Metal shader which looks straightforward, but i'm unsure how to go about knocking out the floor to show everything aside from the shadow.
Here's an example of a shadow catcher: https://knowledge.autodesk.com/search-result/caas/sfdcarticles/sfdcarticles/Maya-2015-Shadow-Catching-with-Use-Background-material.html
Ok, so check out this example project on Github:
https://github.com/carolight/Metal-Shadow-Map/blob/master/Shadows/Shader.metal#L67
That link is to the shader that performs the shadow map testing. Basically, it normalizes the Z position of the fragment being rendered and compares it to the Z-Buffer of the shadow map. If it is less than the shadow-Z, the pixel is fully lit (line 67) else it is tinted slightly (line 69).
What you want to do instead is write (0,0,0, shadow_opacity) for line 69 and (0,0,0,0) for line 67. This should emit transparent pixels. Set shadow_opacity as a uniform from [0.0..1.0].
The rest of the setup is shown in the example. Tilt the plane/camera to the desired setting, skip drawing the cube/vase/whatever in the main pass if you don't really want it in the scene (line 279). (However, note the shadow pass and keep it there).