I am working on an AR project with Swift for iOS. I have some images coming from server, which then turned into ARReferenceImages. With those, I am able to track whatever image I want, even multiple images if I want to.
Problem is, I am rendering a cube on the image when the anchor is placed or updated and this happens with a lower frequency than I initially expected. I am using an ARSession and camera feed is rendered inside a MTKView. Draw calls are correctly firing at a rate that is close to 60 (or close to 60) FPS rate. And I must note that within each draw call I am executing a massive chunk of code (part on GPU and part on CPU) and even that doesn't effect draw time too much. But session(_:didUpdate:) is called very slowly, I have
func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
self.handleTrackedImage(anchors: anchors, isUpdate: false)
}
func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {
self.handleTrackedImage(anchors: anchors, isUpdate: true)
}
and inside handleTrackedImage I am doing some additional stuff (like printing anchors to console and sending them to my server). On console I am able to see that these updates are happening with 0.5 to 1 second delays. I need them to happen with a frequency as close to the camera FPS as possible.
I am rather new to the AR world and I am unable to find anyone who wants to speed up the ARSession like this, there are only questions where people wanted to slow it down. So, what am I missing here?
Turns out if you don't provide a value for maximumNumberOfTrackedImages property of your session configuration, anchor update functions of your delegate won't run on par with your frame speed. If you provide anything other than zero, it will become way more faster and as a result your anchor movement will become much smoother.