Getting the current visible entities in RealityKit

Question

Currently, RealityKit doesn't have any method that provides the currently visible entities. In SceneKit we do have a method for that particular functionality—nodesInsideFrustum(pointOfView).

Our internal solution is to create a big fake bounding box in front of the camera. We then check intersections between the "frustum" bounding box and each entity's bounding box. That, of course, is a bit cumbersome and inaccurate. I wonder if someone can come up with a better solution who is willing to share it.

Answer 1

You could combine two ARView methods:

• ARView.project(position) to get the 2D point in screen space
• ARView.bounds.contains(point) to know if it's visible on screen

But it's not enough, you also have to check if the object is behind you:

• Entity.position(relativeTo: cameraAnchor) (with cameraAnchor being an AnchorEntity(.camera)) to have the local position
• the sign of localPosition.z shows if it's in front or behind the camera