Slow down the ARSession without slowing down the camera feed

Question

I would like to slow down the movement of the 3D augmented object during screen recording without effecting the camera feed. Matrix like effect, is it possible?

I have tried arView.session.pause() but it also froze the camera feed. Can I manage the needed rendered frames? in that case I can just drop the fps on the 3D render, not sure if there is a better way to do it. thanks for any advice.

Answer 1

About ARSession

There is no need to change the speed of the ARSession, as this not only spoil the desired effect, but also ruin the user's AR experience. The session must be running at 60 fps, it must continue to track all the anchors in the scene and mustn't stop.

Freezing animation in RealityKit 2.0

A robust solution would be to use 2 different animation speeds - normal animation speed when you aren't recording and bullet-time animation speed (or even freeze animation) during screen recording.

var speed: Float { get set }               // Default value is 1.0

The "freezing" effect can be achieved using AnimationPlaybackController:

import RealityKit
import ReplayKit

var ctrl: AnimationPlaybackController!

let neo = try ModelEntity.load(named: "Neo_with_Animation")

ctrl = neo.playAnimation(neo.availableAnimations[0].repeat(count: 50), 
                             transitionDuration: 2, 
                             startsPaused: false)

func startRecording(sender: UIButton!) {
    ctrl.speed = 0.02                      // animation speed is 2%
    // some code to start recording...
}
func stopRecording(sender: UIButton!) {
    ctrl.speed = 1.0                       // animation speed is 100%
    ctrl.speed = -1.0                      // reverse animation speed is 100%
    // some code to stop recording...
}

If you need more info on asset animation, read this post.

Freezing physics in RealityKit 2.0

When you're simulating physics you can stop the process using .static case of PhysicsBodyMode enum and resume the process using .dynamic case. However, the process of freezing the physics does not look as impressive as freezing the animation.

let neoScene = try! Experience.loadNeoWithPhysics()

let neo = neoScene.developer!.children[0] as? (ModelEntity & 
                                               HasCollision & 
                                               HasPhysicsBody & 
                                               HasPhysicsMotion)

func startRecording(sender: UIButton!) {
    neo.physicsBody?.mode = .static       // freeze simulation
    // some code to start recording...
}
func stopRecording(sender: UIButton!) {
    neo.physicsBody?.mode = .dynamic      // resume simulation

    // it's quite possible that after unfreezing  
    // an additional impulse will be required...
    neo.physicsMotion?.linearVelocity.x = 1.0
    neo.physicsMotion?.linearVelocity.z = 0.5

    // some code to stop recording...
}

As far as I know, there's no parameter in RealityKit 2.0 that helps you just to slow down (not freeze!) physics simulation without breaking it. Because RealityKit's engine calculates physics in realtime.

Read this post to find out how to quickly setup physics in RealityKit.

Freezing physics in SceneKit

Nonetheless, SceneKit has such a property that changes simulation's rate. Its name is speed.

var speed: CGFloat { get set }