When I went through the video for session 805, Creating Great AR Experiences, and they mention using screen space text when displaying text in AR. They use the example of their Measure app that works quite well.
I want to achieve similar functionality. But, I am not able to figure it out.
Screen Space is the 2D pixel coordinate system of your display. It's where you normally do things like laying out UIKit widgets in screen space, etc.
The alternative to Screen Space is a 3D coordinate space that relates to the 3D world tracking ARKit does or the 3D rendering engine you use to display virtual 3D content in AR. (In that case there are multiple 3D spaces to consider, like "absolute" world space vs camera-relative positioning.)
As mentioned in the WWDC18 talk Creating Great AR Experiences, screen space is a great way to display content that's inherently 2D (like text) in a way that remains readable regardless of the distance to or orientation of a real-world feature that content is associated with. (If you were to place that text label in world space, it'd get small and unreadable as you get far away.)
ARKit doesn't give you any automatic way to have 2D screen-space content follow a 3D world-space position, but making that happen yourself isn't especially difficult. Here's a general recipe for overlaying UIView
s that follow 3D positions:
ARAnchor
at the real-world position where you want screen-space content to appear.renderer(_:didUpdate:for:)
or renderer(_:updateAtTime:)
if you're using SceneKit with ARSCNView
, session(_:didUpdate:)
if you're building your own renderer) to get the current 3D position of that anchor.ARSCNView
projectPoint
, ARCamera
projectPoint
) to get the 2D point where that 3D position appears on screen.UIView
to match that point.For more advanced cases like how the Measure app makes its screen-space label parallel to the line between two 3D points, you'd need to project the two endpoints and use a function like atan2
to get an angle for rotating the label.