Search code examples
iosavfoundationavcapturesessionavcapturedeviceavcapture

Sound not working for some AVCaptureDeviceFormat... is this a bug?


I see this question on SO about the same problem but in my case it is slightly difference.

On that question, the poster says he cannot record audio when he setups the app to shoot video at the highest resolutions a camera can provide under AVFoundation.

On the original question the poster mentions that his AVCaptureConnection has no audio. I believe he is talking inside captureOutput:didOutputSampleBuffer:fromConnection: but in my case the problem is slightly different. In my case, this method is never called for audio. I mean, every time this method is called connection is always a video one... or in other words, there is no data audio output delegate being called here.

I have checked the captureSession and the microphone is there, so captureSession contains a AVCaptureDeviceInput of audio.

(lldb) po _captureSession.inputs
<__NSArrayI 0x170227e00>(
<AVCaptureDeviceInput: 0x17422e2e0 [Back Camera]>,
<AVCaptureDeviceInput: 0x17422e8e0 [iPad Microphone]>
)

I am testing this on an iPad Pro 9.7. I have checked all resolutions of the front and back camera of this device and I have no audio for these:

  1. FRONT CAMERA: 960p @ 30 or 60 fps
  2. BACK CAMERA 4032x3024 at 30 fps

I have tried to remove and add the audio device after changing the resolution but the captureSession hangs and the preview freezes. The app continues to work, no crash, but the preview freezes.

Is this a bug? I don't see any mention on any documentation saying I cannot record audio with the highest resolutions a camera can provide.


NOTE: To demo the problem, I have uploaded a modified version of Apple's CIFunHouse here. I have adjusted line 459 of FHViewController.m with 4032x3024 that is the maximum resolution of my iPad. You should adjust that for the maximum resolution of your device's rear camera.

For some strange reason, when you do that, the app crashes when it tries to initialize the audio. My code, that is based on that, initializes ok but does not record sound. I left the code crashing because perhaps it can help more that way. You will see that channelLayoutand and basicDescription are both NULL for that video format. Reduce the resolution and the audio will initialize ok.


Solution

  • Here is a hand-waving answer: 4032x3024 is not a commonly encountered video resolution. 480p, 720p and 1080p are though. And if you read about 4K resolution video you'll see that 3840x2160 is too.

    In fact "2160p" does capture both audio and video on my iPhone 6s, so why not try that?

    Will AVAssetWriter be able to encode 2160p? Who knows? Maybe.

    But don't be too harsh on AVFoundation - it does a valiant job of putting a sane face on the craziness of hardware. If anything you should log functionality and documentation bugs.