AudioBufferList for AUHAL unit whose output stream format is compressed.

Question

ACKNOWLEDGEMENTS

I know this post is quite long, but I try to contextualise as much as possible my issue because I reckon it's quite unique (couldn't find any related issue apart from this one. The final question is at the very end of the post and here's the complete code.

First of all, a bit of context. I am using CoreAudio and AudioToolbox library, more precisely Audio Units. I am on macOS. My ultimate goal is to record audio from any input device (hence the use of Audio Units over a simple AudioQueueBuffer) and write it to an audio file. I reckon the trickiest part of my program is to convert from LPCM to AAC (in my case) within a single audio unit, hence no use of AUGraph is made.

My program is basically just one Audio Unit, encapsulated in a class, AudioUnit mInputUnit which is an AUHAL unit. Hence, I have followed this this technical note to set it up. Basically, I link the input scope of the input element (since the output element is disabled) to an Audio Device, i.e. my built-in microphone.

Then I update the AudioFormat of the output scope of the unit accordingly.

  ...
  inputStream.mFormatID = kAudioFormatMPEG4AAC;
  inputStream.mFormatFlags = 0;
  inputStream.mBitsPerChannel = 0;
  checkError(
    AudioUnitSetProperty(
      mInputUnit,
      kAudioUnitProperty_StreamFormat,
      kAudioUnitScope_Output,
      1,
      &inputStream,
      propertySize
    ),
    "Couldn't set output stream format."
  );

Therefore, at this point, the Audio Unit should work as follow:

Record from input device in LPCM [INPUT SCOPE] ==> Convert from LPCM to ==> Render in AAC.

Please note that each stream format (input and output) uses 2 channels. Neither the input nor the output stream has its mFormatFlags set to kAudioFormatIsNonInterleaved, so both of them are interleaved. In fact, I think this is where the issue comes from but can't see why.

At this point, everything seems to work right. The issue arises when I try to render the audio unit after having set the input callback.

I have found a note that says the following :

“By convention, AUHAL deinterleaves multichannel audio. This means that you set up two AudioBuffers of one channel each instead of setting up one AudioBuffer with mNumberChannels==2. A common cause of paramErr (-50) problems in AudioUnitRender() calls is having AudioBufferLists whose topology (or arrangement of buffers) doesn’t match what the unit is prepared to produce. When dealing at the unit level, you almost always want to do noninterleaved like this.”

Excerpt From: Chris Adamson & Kevin Avila. “Learning Core Audio: A Hands-On Guide to Audio Programming for Mac and iOS.” iBooks.

Hence, I have followed the appropriate code structure to render the audio.

OSStatus Recorder::inputProc(
  void *inRefCon,
  AudioUnitRenderActionFlags *ioActionFlags,
  const AudioTimeStamp *inTimeStamp,
  UInt32 inBusNumber,
  UInt32 inNumberFrames,
  AudioBufferList *ioData
)
{
  Recorder *This = (Recorder *) inRefCon;
  CAStreamBasicDescription outputStream;
  This->getStreamBasicDescription(kAudioUnitScope_Output, outputStream);

  UInt32 bufferSizeBytes = inNumberFrames * sizeof(Float32);
  UInt32 propertySize = offsetof(AudioBufferList, mBuffers[0]) + (sizeof(AudioBuffer) * outputStream.mChannelsPerFrame);
  auto bufferList = (AudioBufferList*) malloc(propertySize);
  bufferList->mNumberBuffers = outputStream.mChannelsPerFrame;

  for(UInt32 i = 0; i < bufferList->mNumberBuffers; ++i)
  {
    bufferList->mBuffers[i].mNumberChannels = 1;
    bufferList->mBuffers[i].mDataByteSize = bufferSizeBytes;
    bufferList->mBuffers[i].mData = malloc(bufferSizeBytes);
  }

  checkError(
    AudioUnitRender(
      This->mInputUnit,
      ioActionFlags,
      inTimeStamp,
      inBusNumber,
      inNumberFrames,
      bufferList
    ),
    "Couldn't render audio unit."
  );
  free(bufferList);
}

And then, when I try to render the audio, I have the following error Error: Couldn't render audio unit. (-50) which is actually the one that should had been fixed by following the note, which confuses me even more.

THE question

At this point, I don't know if this is something to do with my overall architecture, i.e. should I use a AUGraph and add an output unit instead of trying to convert from canonical format to a compressed format WITHIN a single AUHAL unit? Or is this something to do with the way I pre-allocate my AudioBufferList?

Answer 1

I have managed to fix this issue by redesigning the whole process. To make it short, I still have a unique AUHAL unit, but instead of doing the format conversion within the AUHAL unit, I do it in the render callback, with an Extended Audio File, which takes a source format and a destination format. The whole challenge is to find the right format description, which is basically just testing different values for mFormatID, mFormatFlags etc...