Search code examples
iosswiftscenekitarkitrealitykit

How can I get an entity to constantly look at the camera in RealityKit similar to Billboard Constraint from SceneKit


I have an app that I am trying to update from SceneKit to RealityKit, and one of the features that I am having a hard time replicating in RealityKit is making an entity constantly look at the camera. In SceneKit, this was accomplished by adding the following billboard constraints to the node:

let billboardConstraint = SCNBillboardConstraint()

billboardConstraint.freeAxes = [.X, .Y]
startLabelNode.constraints = [billboardConstraint]

Which would allow the startLabelNode to freely rotate so that it was constantly facing the camera without the startLabelNode changing its position.

However, I can't seem to figure out a way to do this with RealityKit. I have tried the "lookat" method, which doesn't seem to offer the ability to constantly face the camera. Here is a short sample app where I have tried to implement a version of this in RealityKit, but it doesn't offer the ability to have the entity constantly face the camera like it did in SceneKit:

import UIKit
import RealityKit
import ARKit

class ViewController: UIViewController, ARSessionDelegate {
    @IBOutlet weak var arView: ARView!
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        arView.session.delegate = self
        arView.environment.sceneUnderstanding.options = []
        arView.debugOptions.insert(.showSceneUnderstanding) // Display a debug visualization of the mesh.
        arView.renderOptions = [.disablePersonOcclusion, .disableDepthOfField, .disableMotionBlur] // For performance, disable render options that are not required for this app.
        arView.automaticallyConfigureSession = false
        let configuration = ARWorldTrackingConfiguration()
        if ARWorldTrackingConfiguration.supportsSceneReconstruction(.mesh) {
            configuration.sceneReconstruction = .mesh
        } else {
            print("Mesh Classification not available on this device")
            configuration.worldAlignment = .gravity
            configuration.planeDetection = [.horizontal, .vertical]
        }
        configuration.environmentTexturing = .automatic
        arView.session.run(configuration)
        
        UIApplication.shared.isIdleTimerDisabled = true // Prevent the screen from being dimmed to avoid interrupting the AR experience.
    }

    @IBAction func buttonPressed(_ sender: Any) {
        let screenWidth = arView.bounds.width
        let screenHeight = arView.bounds.height
        let centerOfScreen = CGPoint(x: (screenWidth / 2), y: (screenHeight / 2))
        
        if let raycastResult = arView.raycast(from: centerOfScreen, allowing: .estimatedPlane, alignment: .any).first
        {
            addStartLabel(at: raycastResult.worldTransform)
        }
    }
    
    func addStartLabel(at result: simd_float4x4) {
        let resultAnchor = AnchorEntity(world: result)
        resultAnchor.addChild(clickToStartLabel())
        arView.scene.addAnchor(resultAnchor)
    }
    
    func clickToStartLabel() -> ModelEntity {
        let text = "Click to Start Here"
        let textMesh = MeshResource.generateText(text, extrusionDepth: 0.001, font: UIFont.boldSystemFont(ofSize: 0.01))
        let textMaterial = UnlitMaterial(color: .black)
        let textModelEntity = ModelEntity(mesh: textMesh, materials: [textMaterial])
        textModelEntity.generateCollisionShapes(recursive: true)
        textModelEntity.position.x -= textMesh.width / 2
        textModelEntity.position.y -= textMesh.height / 2
        
        let planeMesh = MeshResource.generatePlane(width: (textMesh.width + 0.01), height: (textMesh.height + 0.01))
        let planeMaterial = UnlitMaterial(color: .white)
        let planeModelEntity = ModelEntity(mesh: planeMesh, materials: [planeMaterial])
        planeModelEntity.generateCollisionShapes(recursive:true)
        // move the plane up to make it sit on the anchor instead of in the middle of the anchor
        planeModelEntity.position.y += planeMesh.height / 2
        planeModelEntity.addChild(textModelEntity)
        
        // This does not always keep the planeModelEntity facing the camera
        planeModelEntity.look(at: arView.cameraTransform.translation, from: planeModelEntity.position, relativeTo: nil)
                
        return planeModelEntity
    }
}

extension MeshResource {
    var width: Float
    {
        return (bounds.max.x - bounds.min.x)
    }

    var height: Float
    {
        return (bounds.max.y - bounds.min.y)
    }
}

Is the lookat function the best way to get the missing feature working in RealityKit or is there a better way to have a Entity constantly face the camera?


Solution

  • I was able to figure out an answer to my question. Adding the following block of code allowed the entity to constantly look at the camera:

    func session(_ session: ARSession, didUpdate frame: ARFrame) {
        planeModelEntity.look(at: arView.cameraTransform.translation, from: planeModelEntity.position(relativeTo: nil), relativeTo: nil)
    }