Using both horizontal and vertical planes in ARKit / RealityKit

Question

I’m having trouble with detecting both horizontal and vertical planes in ARKit. I have so far successfully been able to detect horizontal planes to some extent, but would like to integrate this with support for vertical planes as well.

My goal is to create an AR project that has a ball that can bounce off the floor and off walls. Without vertical plane detection, the ball bounces into the distance until it can no longer be seen.

What I have for horizontal plane detection is below.

public func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
    guard let planeAnchor = anchor as? ARPlaneAnchor else { return }
    let width = CGFloat(planeAnchor.extent.x)
    let height = CGFloat(planeAnchor.extent.z)
    let plane = SCNPlane(width: width, height: height)
    plane.materials.first?.diffuse.contents = UIColor.init(white: 1, alpha: 1)
    let planeNode = SCNNode(geometry: plane)
    let x = CGFloat(planeAnchor.center.x)
    let y = CGFloat(planeAnchor.center.y)
    let z = CGFloat(planeAnchor.center.z)
    planeNode.position = SCNVector3(x,y,z)
    planeNode.eulerAngles.x = -.pi / 2
    scene.rootNode.addChildNode(planeNode)
    let box = SCNBox(width: CGFloat(planeAnchor.extent.x), height: CGFloat(planeAnchor.extent.z), length: CGFloat(planeAnchor.extent.y), chamferRadius: 0)
    planeNode.physicsBody = SCNPhysicsBody(type: .static, shape: SCNPhysicsShape(geometry: box, options: nil))
}

How can I have both horizontal and vertical planes in an ARKit or RealityKit project?

Answer 1

ARKit 6.0 + SceneKit for iOS

You can do it using the following approach:

override func viewWillAppear(_ animated: Bool) {
    super.viewWillAppear(animated)

    let configuration = ARWorldTrackingConfiguration()
    configuration.planeDetection = [.horizontal, .vertical]
    arView.session.run(configuration)
}

planeDetection instance property is gettable and settable:

var planeDetection: ARWorldTrackingConfiguration.PlaneDetection { get set }

...and PlaneDetection struct conforms to OptionSet protocol which allows you to use just a single value or both values at the same time:

configuration.planeDetection = []
configuration.planeDetection = .vertical
configuration.planeDetection = [.horizontal, .vertical]

Then add renderer(_:didAdd:for:) instance method to your code that might look like this:

func renderer(_ renderer: SCNSceneRenderer,
             didAdd node: SCNNode,
              for anchor: ARAnchor) {
    
    guard let planeAnchor = anchor as? ARPlaneAnchor 
    else { return }

    let width = CGFloat(planeAnchor.extent.x)
    let height = CGFloat(planeAnchor.extent.z)
    let myPlane = SCNPlane(width: width, height: height)           
    
    if planeAnchor.alignment == .horizontal {
        myPlane.materials.first?.diffuse.contents = UIColor.semiOpaqueRed
    } else if planeAnchor.alignment == .vertical {
        myPlane.materials.first?.diffuse.contents = UIColor.semiOpaqueCyan
    }
    
    let planeNode = SCNNode(geometry: myPlane)
    let x = CGFloat(planeAnchor.center.x)
    let y = CGFloat(planeAnchor.center.y)
    let z = CGFloat(planeAnchor.center.z)
    
    planeNode.position = SCNVector3(x, y, z)
    planeNode.eulerAngles.x = -.pi / 2
    
    node.addChildNode(planeNode)
}

And, at last, add renderer(_:didUpdate:for:) instance method that might look like this:

func renderer(_ renderer: SCNSceneRenderer,
          didUpdate node: SCNNode,
              for anchor: ARAnchor) {
    
    guard let planeAnchor = anchor as? ARPlaneAnchor,
          let planeNode = node.childNodes.first,
          let myPlane = planeNode.geometry as? SCNPlane
    else { return }
    
    let width = CGFloat(planeAnchor.extent.x)
    let height = CGFloat(planeAnchor.extent.z)
    myPlane.width = width
    myPlane.height = height
    
    let x = CGFloat(planeAnchor.center.x)
    let y = CGFloat(planeAnchor.center.y)
    let z = CGFloat(planeAnchor.center.z)
    planeNode.position = SCNVector3(x, y, z)
}

---

---

RealityKit 3.0 for iOS

It only requires just a few lines of code to implement this functionality in RealityKit.

let arView = ARView(frame: .zero)
let model = try! ModelEntity.load(named: "Model.usdz")

let anchor = AnchorEntity(plane: [.horizontal, .vertical], 
                 classification: [.floor, .wall], 
                  minimumBounds: [0.5, 0.5])

anchor.addChild(model)
arView.scene.anchors.append(anchor)

---

---

ARKit 7.0 for visionOS 2.0

The new ARKit API for visionOS 2.0 also gives a developer the ability to detect and track sloping surfaces (.slanted case) that are neither horizontal nor vertical.

let planeDetection = PlaneDetectionProvider(alignments: [.horizontal, 
                                                         .vertical, 
                                                         .slanted])