ARKit3 - Official Apple example won't compile, has flawed USDZ 3D mesh/skeleten model

Question

I am investigating new ARKit3 features, in particular motion capture. I have an iPhone with A12 chip (so all new features should work), and iPhone is loaded with iOS 13 (beta). I also installed the Xcode 11 beta on my development laptop as recommended.

When I download the tutorial / sample Xcode project here I find that I have errors on compile. I was able to get rid of those by commenting out the references to the AnyCancellable instance, and the program compiles.

When I run it on on my device, I get error messages about the 3D mesh (in USDZ format) saying it is missing certain joint information.

I've tried substituting the USDZ model included with the sample project for other USDZ models provided on the apple site here, to no avail.

The expected behaviour is that the sample app should open in a camera view, track a person that appears in front of the camera and render a skeleton with 3D mesh model overtop, which mimics the person's actions.

I am getting the error in the Xcode console:

2019-07-18 16:21:13.645289-0400 BodyDetection[11503:176170] [API] Created entity character does not contain supplied joint name: right_shoulder_2_joint.
2019-07-18 16:21:13.645348-0400 BodyDetection[11503:176170] [API] Cannot get joint count for non-character entity.
Error: Unable to load model: The operation couldn’t be completed. (RealityKit.Entity.LoadError error 4.)

It seems it cannot load the mesh / skeleton. The app runs and I can see the camera view, but no 3D character/mesh is ever rendered.

Can anyone give me a hint as to where to find a 3D model that will work with the ARKit3 framework? Has anyone seen anything like this error before?

Answer 1

Can you confirm that your device definitely has an A12 chip (meaning it's an iPhone XR, XS, XS Max, iPad Mini (2019), iPad Air (2019), or iPad Pro (2018))? Additionally, ensure your Xcode is running the latest Beta 4 release (build 11M374r, as of the time of this writing) and your iOS device is running the iOS 13 beta.

These may seem rudimentary, but I cannot replicate the issues you indicate when downloading the sample project on my end. I can launch the project, set my signing team, load on my iPhone XS Max, and the project functions as it should; when a body is detected in the frame, the 3D "skeleton" appears alongside and motion follows.

It may be worth mentioning that the 3D body tracking technology appears to require a USDZ model that has existing joints already configured. The USDZ models on Apple's site mostly do not (the robot, for example, lacks any such joints that could be tracked). See this screenshot for an example of what the structure of Apple's "skeleton" looks like.

The error messages you provided from Xcode make it seem like the model you are trying to load lacks the skeletal structure to conform to this technology. Could you try re-downloading the sample project and confirm you get the same error?