CMSampleBuffer frame converted to vImage has wrong colors

Question

I’m trying to convert CMSampleBuffer from camera output to vImage and later apply some processing. Unfortunately, even without any further editing, frame I get from buffer has wrong colors:

Implementation (Memory management and errors are not considered in question):

Configuring video output device:

    videoDataOutput = AVCaptureVideoDataOutput()
    videoDataOutput.videoSettings = [String(kCVPixelBufferPixelFormatTypeKey): kCVPixelFormatType_32BGRA]
    videoDataOutput.alwaysDiscardsLateVideoFrames = true
    videoDataOutput.setSampleBufferDelegate(self, queue: captureQueue)
    videoConnection = videoDataOutput.connection(withMediaType:  AVMediaTypeVideo)

    captureSession.sessionPreset = AVCaptureSessionPreset1280x720

    let videoDevice = AVCaptureDevice.defaultDevice(withMediaType:  AVMediaTypeVideo)
    guard let videoDeviceInput = try? AVCaptureDeviceInput(device: videoDevice) else {
        return
    }

Creating vImage from CASampleBuffer received from camera:

   // Convert `CASampleBuffer` to `CVImageBuffer`
    guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return }

    var buffer: vImage_Buffer = vImage_Buffer()
    buffer.data = CVPixelBufferGetBaseAddress(pixelBuffer)
    buffer.rowBytes = CVPixelBufferGetBytesPerRow(pixelBuffer)
    buffer.width = vImagePixelCount(CVPixelBufferGetWidth(pixelBuffer))
    buffer.height = vImagePixelCount(CVPixelBufferGetHeight(pixelBuffer))

    let vformat = vImageCVImageFormat_CreateWithCVPixelBuffer(pixelBuffer)
    let bitmapInfo:CGBitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.last.rawValue | CGBitmapInfo.byteOrder32Little.rawValue)

    var cgFormat = vImage_CGImageFormat(bitsPerComponent: 8,
                                        bitsPerPixel: 32,
                                        colorSpace: nil,
                                        bitmapInfo: bitmapInfo,
                                        version: 0,
                                        decode: nil,
                                        renderingIntent: .defaultIntent)

     // Create vImage
     vImageBuffer_InitWithCVPixelBuffer(&buffer, &cgFormat, pixelBuffer, vformat!.takeRetainedValue(), cgColor, vImage_Flags(kvImageNoFlags))

Converting buffer to UIImage:

For the sake of tests CVPixelBuffer is exported to UIImage, but adding it to video buffer has the same result.

var dstPixelBuffer: CVPixelBuffer?

    let status = CVPixelBufferCreateWithBytes(nil, Int(buffer.width), Int(buffer.height),
                                              kCVPixelFormatType_32BGRA, buffer.data,
                                              Int(buffer.rowBytes), releaseCallback,
                                              nil, nil, &dstPixelBuffer)

    let destCGImage = vImageCreateCGImageFromBuffer(&buffer, &cgFormat, nil, nil, numericCast(kvImageNoFlags), nil)?.takeRetainedValue()

    // create a UIImage
    let exportedImage = destCGImage.flatMap { UIImage(cgImage: $0, scale: 0.0, orientation: UIImageOrientation.right) }

    DispatchQueue.main.async {
        self.previewView.image = exportedImage
    }

Answer 1

Try setting the color space on your CV image format:

    let vformat = vImageCVImageFormat_CreateWithCVPixelBuffer(pixelBuffer).takeRetainedValue()

    vImageCVImageFormat_SetColorSpace(vformat,
                                      CGColorSpaceCreateDeviceRGB())

...and update your call to vImageBuffer_InitWithCVPixelBuffer to reflect the fact vformat is now a managed reference:

    let error = vImageBuffer_InitWithCVPixelBuffer(&buffer, &cgFormat, pixelBuffer, vformat, nil, vImage_Flags(kvImageNoFlags))

Finally, your can remove the following lines, vImageBuffer_InitWithCVPixelBuffer is doing that work for you:

//        buffer.data = CVPixelBufferGetBaseAddress(pixelBuffer)
//        buffer.rowBytes = CVPixelBufferGetBytesPerRow(pixelBuffer)
//        buffer.width = vImagePixelCount(CVPixelBufferGetWidth(pixelBuffer))
//        buffer.height = vImagePixelCount(CVPixelBufferGetHeight(pixelBuffer))

Note that you don't need to lock the Core Video pixel buffer, if you check the headerdoc, it says "It is not necessary to lock the CVPixelBuffer before calling this function".