Get RGB average of "CIAreaAverage" from CMSampleBuffer in Float precision in Swift

Question

I am trying to get the average RGB value for my "AVCaptureVideoDataOutput" feed. I found the following solution on StackOverflow:

let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)
let cameraImage = CIImage(CVPixelBuffer: pixelBuffer!)
let filter = CIFilter(name: "CIAreaAverage")
filter!.setValue(cameraImage, forKey: kCIInputImageKey)
let outputImage = filter!.valueForKey(kCIOutputImageKey) as! CIImage!

let ctx = CIContext(options:nil)
let cgImage = ctx.createCGImage(outputImage, fromRect:outputImage.extent)

let rawData:NSData = CGDataProviderCopyData(CGImageGetDataProvider(cgImage))!
let pixels = UnsafePointer<UInt8>(rawData.bytes)
let bytes = UnsafeBufferPointer<UInt8>(start:pixels, count:rawData.length)
var BGRA_index = 0
for pixel in UnsafeBufferPointer(start: bytes.baseAddress, count: bytes.count) {
     switch BGRA_index {
         case 0:
              bluemean = CGFloat (pixel)
         case 1:
              greenmean = CGFloat (pixel)
         case 2:
              redmean = CGFloat (pixel)
         case 3:
              break
         default:
              break
     }
     BGRA_index++
}

But this produces the average as an Int but I need it in a Float format with the precision kept. The rounding is quite problematic in the problem domain I'm working with. Is there a way to a Float average efficiently?

Thanks a lot!

Answer 1

May I recommend using our library CoreImageExtensions for reading the value? We added methods for reading pixel values from CIImages in different formats. For your case it would look like this:

 import CoreImageExtensions

 let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)
 let cameraImage = CIImage(cvPixelBuffer: pixelBuffer!)
 let filter = CIFilter(name: "CIAreaAverage")!
 filter.setValue(cameraImage, forKey: kCIInputImageKey)
 filter.setValue(CIVector(cgRect: cameraImage.extent), forKey: kCIInputExtentKey)
 let outputImage = filter.outputImage!

 let context = CIContext()
 // get the value of a specific pixel as a `SIMD4<Float32>`
 let average = context.readFloat32PixelValue(from: outputImage, at: CGPoint.zero)

Also keep in mind, if you want to compute the average regularly (not just once), to only create a single instance of CIContext and reuse it for every camera frame. Creating it is expensive and it actually increases performance to use the same instance since it caches internal resources.