Distorted audio when playing raw PCM data buffer in iOS

Question

I'm trying to play audio buffers using Audio Unit in iOS. The audio buffer is from a C library which receives the audio from Playstation 4 and decode it using Opus. The format of the audio buffers is PCM int16_t.

With Audio Unit and TPCircularBuffer, I am done getting it to play the sound. However it is badly distorted and not clean.

Here is the setup of my Audio Unit class

init() {
    _TPCircularBufferInit(&buffer, 960
                          , MemoryLayout<TPCircularBuffer>.size)
    setupAudio()
}

func play(data: NSMutableData) {
    TPCircularBufferProduceBytes(&buffer, data.bytes, UInt32(data.length))
}

private func setupAudio() {
    do {
        try AVAudioSession.sharedInstance().setActive(true, options: [])
    } catch { }
    
    var audioComponentDesc = AudioComponentDescription(componentType: OSType(kAudioUnitType_Output),
                                                       componentSubType: OSType(kAudioUnitSubType_RemoteIO),
                                                       componentManufacturer: OSType(kAudioUnitManufacturer_Apple),
                                                       componentFlags: 0, componentFlagsMask: 0)
    let inputComponent = AudioComponentFindNext(nil, &audioComponentDesc)
    status = AudioComponentInstanceNew(inputComponent!, &audioUnit)
    
    if status != noErr {
        print("Audio Component Instance New Error \(status.debugDescription)")
    }
    
    var audioDescription = AudioStreamBasicDescription()
    audioDescription.mSampleRate = 48000
    audioDescription.mFormatID = kAudioFormatLinearPCM
    audioDescription.mFormatFlags = kAudioFormatFlagIsPacked | kAudioFormatFlagIsSignedInteger
    audioDescription.mChannelsPerFrame = 2
    audioDescription.mFramesPerPacket = 1
    audioDescription.mBitsPerChannel = 16
    audioDescription.mBytesPerFrame = (audioDescription.mBitsPerChannel / 8) * audioDescription.mChannelsPerFrame
    audioDescription.mBytesPerPacket = audioDescription.mBytesPerFrame * audioDescription.mFramesPerPacket
    audioDescription.mReserved = 0
    status = AudioUnitSetProperty(audioUnit, kAudioUnitProperty_StreamFormat,
                                  kAudioUnitScope_Input, 0, &audioDescription,
                                  UInt32(MemoryLayout<AudioStreamBasicDescription>.size))
    
    if status != noErr {
        print("Enable IO for playback error \(status.debugDescription)")
    }
    
    var flag: UInt32 = 0
    status = AudioUnitSetProperty(audioUnit, kAudioOutputUnitProperty_EnableIO,
                                  kAudioUnitScope_Input, 1, &flag,
                                  UInt32(MemoryLayout.size(ofValue: flag)))
    
    if status != noErr {
        print("Enable IO for playback error \(status.debugDescription)")
    }
    
    var outputCallbackStruct = AURenderCallbackStruct()
    outputCallbackStruct.inputProc = performPlayback
    outputCallbackStruct.inputProcRefCon = UnsafeMutableRawPointer(Unmanaged.passUnretained(self).toOpaque())
    status = AudioUnitSetProperty(audioUnit, kAudioUnitProperty_SetRenderCallback,
                                  kAudioUnitScope_Global, 0, &outputCallbackStruct,
                                  UInt32(MemoryLayout<AURenderCallbackStruct>.size))
    status = AudioUnitInitialize(audioUnit)
    if status != noErr {
        print("Failed to initialize audio unit \(status!)")
    }
    status = AudioOutputUnitStart(audioUnit)
    if status != noErr {
        print("Failed to initialize output unit \(status!)")
    }
}

The playback function

private func performPlayback(clientData: UnsafeMutableRawPointer,_ ioActionFlags: UnsafeMutablePointer<AudioUnitRenderActionFlags>, inTimeStamp: UnsafePointer<AudioTimeStamp>,
                         inBufNumber: UInt32, inNumberFrames: UInt32, ioData: UnsafeMutablePointer<AudioBufferList>?) -> OSStatus {
let player = Unmanaged<AudioPlayer>.fromOpaque(UnsafeRawPointer(clientData)!).takeUnretainedValue()
let buffer = ioData![0].mBuffers
let bytesToCopy = ioData![0].mBuffers.mDataByteSize
var bufferTail: UnsafeMutableRawPointer?
var availableBytes: UInt32 = 0
bufferTail = TPCircularBufferTail(&player.buffer, &availableBytes)
let bytesToWrite = min(bytesToCopy, availableBytes)
memcpy(buffer.mData, bufferTail, Int(bytesToWrite))
TPCircularBufferConsume(&player.buffer, bytesToWrite)
return noErr}

Here is where I call the play function

private func StreamAudioFrameCallback(buffers: UnsafeMutablePointer<Int16>?,
                                  samplesCount: Int, user: UnsafeMutableRawPointer?) {
let decodedData = NSMutableData()
if let buffer = buffers, samplesCount > 0 {
    let decodedDataSize = samplesCount * MemoryLayout<opus_int16>.size
    decodedData.append(buffer, length: decodedDataSize)
    AudioPlayer.shared.play(data: decodedData)
}

Is anyone familiar with this ? Any helps is appreciated.

Answer 1

You might not want to start playing until there's more in the circular buffer than the amount needed to cover the maximum time jitter in the incoming data rate. Try waiting until there's half a second of audio in the circular buffer before starting to play it. Then experiment with the amount of padding.