How do I specify a child node location relative to ARImageAnchor?

Question

I have a RealityKit app that is doing some basic AR image tracking. It detects a rectangular-shaped image, and I am looking to place some spherical dots at each corner of the image. I know I can create the spheres themselves using a ModelEntity, but I haven't been able to figure out how to specify the position of these relative to the established ARImageAnchor from the reference image.

I think I just need a counterpart to SceneKit's addChildNode(SCNNode) function, which uses SCNVector3Make() to specify a position. I just haven't been able to find a way to establish a relative position and assign a child node to the ARImageAnchor outside of these SceneKit functions. Is there something built into RealityKit that would accomplish this, or is there a way to use SceneKit to place the corner dots while still using my current setup with RealityKit for the AR reference image tracking?

Answer 1

Try the following approach:

import ARKit
import RealityKit
 
class ViewController: UIViewController {
    
    @IBOutlet var arView: ARView!
    var anchorEntity = AnchorEntity()
    let model_01 = ModelEntity(mesh: .generateSphere(radius: 0.03))
    let model_02 = ModelEntity(mesh: .generateSphere(radius: 0.03))
    let model_03 = ModelEntity(mesh: .generateSphere(radius: 0.03))
    let model_04 = ModelEntity(mesh: .generateSphere(radius: 0.03))
    
    override func viewWillAppear(_ animated: Bool) {
        super.viewWillAppear(animated)            
        arView.session.delegate = self
        
        guard let reference = ARReferenceImage.referenceImages(
                                                inGroupNamed: "AR Resources",
                                                      bundle: nil)
        else { return }
        
        let config = ARImageTrackingConfiguration()
        config.trackingImages = reference
        arView.session.run(config)
        
        self.anchorEntity.addChild(model_01)
        self.anchorEntity.addChild(model_02)
        self.anchorEntity.addChild(model_03)
        self.anchorEntity.addChild(model_04)    
        arView.scene.anchors.append(self.anchorEntity)
    }
}

Then implement session(_:didUpdate:) method:

extension ViewController: ARSessionDelegate {
    
    func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {
        
        guard let imageAnchor = anchors.first as? ARImageAnchor
        else { return }
        
        let width = Float(imageAnchor.referenceImage.physicalSize.width)
        let height = Float(imageAnchor.referenceImage.physicalSize.height)
        let x = imageAnchor.transform.columns.3.x
        let y = imageAnchor.transform.columns.3.y
        let z = imageAnchor.transform.columns.3.z
        
        let lowerLeft = SIMD3<Float>(x - width/2, y - height/2, z)
        let lowerRight = SIMD3<Float>(x + width/2, y - height/2, z)
        let upperRight = SIMD3<Float>(x + width/2, y + height/2, z)
        let upperLeft = SIMD3<Float>(x - width/2, y + height/2, z)
        
        self.model_01.position = lowerLeft
        self.model_02.position = lowerRight
        self.model_03.position = upperRight
        self.model_04.position = upperLeft
        
        self.anchorEntity = AnchorEntity(anchor: imageAnchor)
    }
}