I have an app that uses ImageTracking to generate 3D models of cars on top of 2D pictures on my table. I want detect when one of the cars has been touched, and do an action, say play an engine sound. Detecting when the users touched the screen and playing a sound is easy, but I can't find out how to play a sound only when the 3D augmented model is clicked. Here is a video of what I want to do: video. This guy does the same thing as me, but when he clicks that generated text, he gets a popup instead of playing an engine noise.
Here is my code so far.
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
let configuration = ARWorldTrackingConfiguration()
if let audi = ARReferenceImage.referenceImages(inGroupNamed: "Audi", bundle: Bundle.main) {
configuration.detectionImages = audi
configuration.maximumNumberOfTrackedImages = 2
print("Image added!")
}
let audi = ARReferenceImage.referenceImages(inGroupNamed: "Audi", bundle: Bundle.main)
// Run the view's session
sceneView.session.run(configuration)
}
Apologies if the "audi" name is misleading. The Audi group is an ARResourceGroup with 2 images, one of an Audi R8, and the other of a Mercedes AMG Coupe. Here is the renderer function to render each car onto its respective picture:
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
let node = SCNNode()
if let imageAnchor = anchor as? ARImageAnchor {
//print(imageAnchor.referenceImage.name)
let plane = SCNPlane(width: imageAnchor.referenceImage.physicalSize.width, height: imageAnchor.referenceImage.physicalSize.height)
plane.firstMaterial?.diffuse.contents = UIColor(white: 1.0, alpha: 0.75)
let planeNode = SCNNode(geometry: plane)
planeNode.eulerAngles.x = -.pi/2
node.addChildNode(planeNode)
if imageAnchor.referenceImage.name! == "mercedes2019coupe" {
let mercedesScene = SCNScene(named: "art.scnassets/mercedesAR.scn")!
let mercedesNode = mercedesScene.rootNode.childNodes.first!
mercedesNode.eulerAngles.x = .pi/2
mercedesNode.eulerAngles.z = -.pi/2
let yShift = (mercedesNode.boundingBox.max.y - mercedesNode.boundingBox.min.y) / 2
mercedesNode.position.y += yShift * 0.05
planeNode.addChildNode(mercedesNode)
}
if imageAnchor.referenceImage.name! == "audir8" {
let audiScene = SCNScene(named: "art.scnassets/audi.scn")!
let audiNode = audiScene.rootNode.childNodes.first!
audiNode.eulerAngles.x = .pi/2
audiNode.eulerAngles.z = -.pi/2
let yShift = (audiNode.boundingBox.max.y - audiNode.boundingBox.min.y) / 2
print(audiNode.boundingBox.max.y)
print(audiNode.boundingBox.min.y)
audiNode.position.y += yShift * 0.1
//audiNode.boundingSphere.radius
planeNode.addChildNode(audiNode)
}
}
return node
}
As you see, I create a plane on top of each image, then generate the 3D car onto each plane when detected. What I can't find out though is how to detect when a certain plane has been touched. Here is the code to play a sound when the user taps the screen:
override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
if let touch = touches.first {
playSound()
//I create the playSound function elsewhere in my code, it works so there is no need to include it in this question
}
}
I've tried out a bunch of forms of the touchesBegan function, and tried out the gesture function, but nothing seems to work. If anyone can help me out, that would be much appreciated. Thanks in advance!
Assuming to understand you correctly, you want to touch the car model by tapping it as seen on screen.
What you probably need is doing a so called hitTest, a SceneKit function to identify geometry by tapping the screen.
Here are two example pages on Stack Overflow. This should bring you in the right direction.
How to do a hitTest on subnodes of node in sceneView?
And the Apple Docs:
https://developer.apple.com/documentation/scenekit/scnscenerenderer/1522929-hittest
https://developer.apple.com/documentation/scenekit/scnhittestresult
(there are a plenty of examples out there how to do this. Also the default, out of the box, Scenekit Template, the one with the airplane, comes with a hittest example - create a new game project with XCode and select Scenekit)