Search code examples
augmented-realityscenekitarkitarcorerealitykit

Augmented Reality – Lighting Real-World objects with Virtual light


Is it possible to import a virtual lamp object into the AR scene, that projects a light cone, which illuminates the surrounding space in the room and the real objects in it, e.g. a table, floor, walls?

For ARKit, I found this SO post.

For ARCore, there is an example of relighting technique. And this source code.

I have also been suggested that post-processing can be used to brighten the whole scene.

However, these examples are from a while ago and perhaps threre is a newer or a more straight forward solution to this problem?


Solution

  • At the low level, RealityKit is only responsible for rendering virtual objects and overlaying them on top of the camera frame. If you want to illuminate the real scene, you need to post-process the camera frame.

    Here are some tutorials on how to do post-processing: Tutorial1⃣️ Tutorial2⃣️


    If all you need is an effect like This , then all you need to do is add a CGImage-based post-processing effect for the virtual object (lights).

    More specifically, add a bloom filter to the rendered image(You can also simulate bloom filters with Gaussian blur).

    In this way, the code is all around UIImage and CGImage, so it's pretty simple😎

    If you want to be more realistic, consider using the depth map provided by LiDAR to calculate which areas can be illuminated for a more detailed brightness.


    Or If you're a true explorer, you can use Metal to create a real world Digital Twin point cloud in real time to simulate occlusion of light.