Search code examples
iosobjective-ciphonesprite-kitcore-audio

Does SpriteKit and SceneKit allow access to iOS microphone input during game play?


I am making a simple sprite kit game that needs an audio input from the microphone for voice commands. I have already created a simple game and I have already created a separate app that if you speak a word it detects and it displays on the screen using cmu-sphinx/pocketsphinx (http://cmusphinx.sourceforge.net/).

I was using Novocaine (https://github.com/alexbw/novocaine) because I have used this library numerous times and had great success with it in other projects in the past, but whenever I initiate the Novocaine object in the game, my app crashes with an exception.

So I figured that since I was using a wrapper library it must have been some deprecated functions that were not supported any more, so I did some more research. I came across Apple's demo app AurioTouch (https://developer.apple.com/library/ios/samplecode/aurioTouch/Introduction/Intro.html). After playing around with that sample app, I was able to narrow down the classes that I needed to get the raw input values, which was primarily the AudioController class. So I imported over the AudioController class and its necessary components and was able to build it. Fortunately, the example had proper exception handling so I was able to see where it was failing. The exception was caused by this line:

AudioUnitInitialize(_rioUnit)

I was beginning to suspect that trying to access the microphone in a SpriteKit/SceneKit environment was causing the issue. I've also tried the same steps mentioned above in brand new SpriteKit and SceneKit projects and did not resolve the issue. So I am wondering if it is anyway possible for you to give real time microphone input within a SpriteKit/SceneKit project.


Solution

  • There is a setting in the iOS Settings app under Privacy : Microphone, where you have to switch on permission for an app to use the device's microphone. The AVAudioSession requestRecordPermission API can help check this permission setting.

    There are also some AVAudioSession categories that should be set and activated to use the microphone. The use of SpriteKit and SceneKit does not interfere with these settings.