Search code examples
iosswiftimage-processingmetalaccelerate

Conversion of kCVPixelFormatType_420YpCbCrBiPlanarVideoRange to kCVPixelFormatType_32BGRA

In an iOS app I am working on, the image is captured in kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange(to be send over in network) but later in the app I want it converted to kCVPixelFormatType_32BGRA (to be used for Deep Learning inference). I am unable to perform this conversion properly. I found some help in the apple documentation @ vImageConvert_420Yp8_Cb8_Cr8ToARGB8888(::::::::); which is a function that

Converts a YUV 420Yp8_Cb8_Cr8 format image to an ARGB8888 format image.

This seems like it would fit perfectly for my use case. But I am unable to figure out how to get the arguments srcYp, srcCb and srcCr. The function definition is:

func vImageConvert_420Yp8_Cb8_Cr8ToARGB8888(_ srcYp: UnsafePointer<vImage_Buffer>, 
                                      _ srcCb: UnsafePointer<vImage_Buffer>, 
                                      _ srcCr: UnsafePointer<vImage_Buffer>, 
                                      _ dest: UnsafePointer<vImage_Buffer>, 
                                      _ info: UnsafePointer<vImage_YpCbCrToARGB>, 
                                      _ permuteMap: UnsafePointer<UInt8>!, 
                                      _ alpha: UInt8, 
                                      _ flags: vImage_Flags) -> vImage_Error

Any help is deeply appreciated.

Solution

  • I think you need vImageConvert_420Yp8_CbCr8ToARGB8888 since the chroma values are in one plane, not two.

    I found this example on how to convert the pixel buffer planes to vImage_Buffers and pass them to the function. Maybe it helps.