Darken/Lighten ARKit camera feed behind objects

Question

I'm currently working on an ARKit project where I would like to darken the actual camera feed so the objects in my 3D scene stand out more.

2 Solutions I found so far:

A) Manually applying CIFilter to the camera frames and setting those as background image to the SceneKit scene as answered in this SO post The problem here is that fps tanks significantly.

B) Set a background color like so:

sceneView.scene.background.contents = UIColor(white: 0.0, alpha: 0.2)

Sadly, colors with alpha <1 are still opaque, so no matter what alpha I set I can't see anything of the camera feed.

Can anyone think of a different trick to darken the camera feed?

Answer 1

Your option B doesn't work for two reasons:

1. the scene view is opaque, so there's nothing behind it for a partially transparent background color to blend with.
2. sceneView.scene.background is what actually displays the camera image, so if you set it to a color you're not displaying the camera image at all anymore.

Some other options (mostly untested) you might look into:

• As referenced from the answer you linked, use SCNTechnique to set up multipass rendering. On the first pass, render the whole scene with the excludeCategoryMask (and your scene contents) set up to render nothing but the background, using a shader that dims (or blurs or whatever) all pixels. On the second pass, render only the node(s) you want to appear without that shader (use a simple pass-through shader).

• Keep your option B, but make the scene view non-opaque. Set the view's backgroundColor (not the scene's background) to a solid color, and set the transparency of the scene background to fade out the camera feed against that color.

• Use geometry to create a "physical" backdrop for your scene — e.g. an SCNPlane of large size, placed as a child of the camera at some fixed distance that's much farther than any other scene content. Set the plane's background color and transparency.

• Use multiple views — the ARSCNView, made non-opaque and with a clear background, and another (not necessarily an SceneKit view) that just shows the camera feed. Mess with the other view (or drop in other fun things like UIVisualEffectView) to obscure the camera feed.

• File a bug with Apple about sceneView.background not getting the full set of shading customization options that nodes and materials get (filters, shader modifiers, full shader programs) etc, without which customizing the background is much more difficult than customizing other aspects of the scene.