Search code examples
swiftaudionsarrayavfoundationnsdata

How to convert Data of Int16 audio samples to array of float audio samples


I'm currently working with audio samples. I get them from AVAssetReader and have a CMSampleBuffer with something like this:

guard let sampleBuffer = readerOutput.copyNextSampleBuffer() else {
guard reader.status == .completed else { return nil }
// Completed
// samples is an array of Int16
let samples = sampleData.withUnsafeBytes {
  Array(UnsafeBufferPointer<Int16>(
  start: $0, count: sampleData.count / MemoryLayout<Int16>.size))
 }

 // The only way I found to convert [Int16] -> [Float]...
 return samples.map { Float($0) / Float(Int16.max)}
}

guard let blockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer) else {
return nil
}

let length = CMBlockBufferGetDataLength(blockBuffer)
let sampleBytes = UnsafeMutablePointer<UInt8>.allocate(capacity: length)
      CMBlockBufferCopyDataBytes(blockBuffer, 0, length, sampleBytes)

      sampleData.append(sampleBytes, count: length)
}

As you can see the only I found to convert [Int16] -> [Float] issamples.map { Float($0) / Float(Int16.max) but by doing this my processing time is increasing. Does it exist an other way to cast a pointer of Int16 to a pointer of Float?


Solution

  • "Casting" or "rebinding" a pointer only changes the way how memory is interpreted. You want to compute floating point values from integers, the new values have a different memory representation (and also a different size).

    Therefore you somehow have to iterate over all input values and compute the new values. What you can do is to omit the Array creation:

    let samples = sampleData.withUnsafeBytes {
        UnsafeBufferPointer<Int16>(start: $0, count: sampleData.count / MemoryLayout<Int16>.size)
    }
    return samples.map { Float($0) / Float(Int16.max) }
    

    Another option would be to use the vDSP functions from the Accelerate framework:

    import Accelerate
    // ...
    
    let numSamples = sampleData.count / MemoryLayout<Int16>.size
    var factor = Float(Int16.max)
    var floats: [Float] = Array(repeating: 0.0, count: numSamples)
    
    // Int16 array to Float array:
    sampleData.withUnsafeBytes {
        vDSP_vflt16($0, 1, &floats, 1, vDSP_Length(numSamples))
    }
    // Scaling:
    vDSP_vsdiv(&floats, 1, &factor, &floats, 1, vDSP_Length(numSamples))
    

    I don't know if that is faster, you'll have to check. (Update: It is faster, as ColGraff demonstrated in his answer.)

    An explicit loop is also much faster than using map:

    let factor = Float(Int16.max)
    let samples = sampleData.withUnsafeBytes {
        UnsafeBufferPointer<Int16>(start: $0, count: sampleData.count / MemoryLayout<Int16>.size)
    }
    var floats: [Float] = Array(repeating: 0.0, count: samples.count)
    for i in 0..<samples.count {
        floats[i] = Float(samples[i]) / factor
    }
    return floats
    

    An additional option in your case might be to use CMBlockBufferGetDataPointer() instead of CMBlockBufferCopyDataBytes() into allocated memory.