Search code examples
realitykitvisionosvision-pro

Movement controls not following user head rotation in RealityKit


I'm implementing movement controls in visionOS using RealityKit. I have a set of movement controls anchored to the user's head position:

@State private var userAnchor = AnchorEntity(.head)

private func createControlEntity(width: Float, height: Float, direction: String) -> ModelEntity {
        let entity = ModelEntity(
            mesh: .generatePlane(width: width * 0.6, height: height * 0.6, cornerRadius: 0.01),
            materials: [SimpleMaterial(color: .white.withAlphaComponent(0.3), isMetallic: false)]
        )
        
        entity.name = "control_\(direction)"
        
        let collisionBox = ShapeResource.generateBox(size: [width * 0.6, height * 0.6, 0.05]) 
        entity.collision = CollisionComponent(
            shapes: [collisionBox],
            mode: .default,
            filter: .default
        )
        
        entity.components[InputTargetComponent.self] = InputTargetComponent(
            allowedInputTypes: [.direct, .indirect]
        )
        
        entity.components[ControlComponent.self] = ControlComponent(direction: direction)
        
        return entity
    }

// Control buttons attached to userAnchor
controlEntities = [
    "forward": createControlEntity(width: 0.04, height: 0.04, direction: "forward"),
    "backward": createControlEntity(width: 0.04, height: 0.04, direction: "backward"),
    // ...
]

The controls follow the user's head position, but when calculating movement direction in updateEntityPosition, the scene doesn't move in the direction the user is facing.

I'm using the head anchor's transform to calculate movement direction:

private func updateEntityPosition(_ entity: Entity) {
    let currentTransform = userAnchor.transform.matrix
    
    let forward = normalize(SIMD3<Float>(
        currentTransform.columns.2.x,  
        0,
        currentTransform.columns.2.z   
    ))
    
    var movement = SIMD3<Float>.zero
    if movementModel.isMovingForward {
        movement -= forward * movementModel.movementSpeed 
    }
    
    let newPosition = entity.position + movement
    entity.move(
        to: Transform(scale: entity.transform.scale,
                     rotation: entity.transform.rotation,
                     translation: newPosition),
        relativeTo: nil,
        duration: 0.05,
        timingFunction: .linear
    )
}

Expected: When the user turns their head and presses forward, the scene should move in their current facing direction. Actual: The scene moves in the original forward direction, ignoring head rotation.


Solution

  • The head anchor transform is hidden by design. See RealityKit track head movement/position? for a way to query device position using WorldTrackingProvider instead.