Search code examples
objective-ccocoaimage-processingvimage

Scale vImage_Buffer with offset - Cocoa Objective C


I am trying to scale an image using vImage_Buffer and the below code works for me. My trouble is I want to maintain the aspect ratio of the source image, so I might need to add a xOffset or yOffset. Below code only works for yOffset. How can I scale the image with xOffset as well. I can not do scaling with CGContext since that affect the performance.

    CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
    
    size_t finalWidth = 1080;
    size_t finalHeight = 720;
    
    size_t sourceWidth = CVPixelBufferGetWidth(imageBuffer);
    size_t sourceHeight = CVPixelBufferGetHeight(imageBuffer);
    
    CGRect aspectRect = AVMakeRectWithAspectRatioInsideRect(CGSizeMake(sourceWidth, sourceHeight), CGRectMake(0, 0, finalWidth, finalHeight));
    
    size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);

    size_t startY = aspectRect.origin.y;
    size_t yOffSet = (finalWidth*startY*4);
    CVPixelBufferLockBaseAddress(imageBuffer, 0);
    
    void *baseAddress = CVPixelBufferGetBaseAddress(imageBuffer);
    
    void* destData = malloc(finalHeight * finalWidth * 4);
    
    vImage_Buffer srcBuffer = { (void *)baseAddress, sourceHeight, sourceWidth, bytesPerRow};
    vImage_Buffer destBuffer = { (void *)destData+yOffSet, aspectRect.size.height, aspectRect.size.width, aspectRect.size.width * 4};
    
    vImage_Error err = vImageScale_ARGB8888(&srcBuffer, &destBuffer, NULL, 0);

Solution

  • No pun intended, but you should really read Accelerate.framework documentation.

    Replace malloc with calloc ...

    void *destData = calloc(finalHeight * finalWidth * 4);
    

    ... to zero all the bytes (or use any other way).

    What does vImage_Buffer.rowBytes documentation say?

    The distance, in bytes, between the start of one pixel row and the next in an image, including any unused space between them.

    The rowBytes value must be at least the width multiplied by the pixel size, where the pixel size depends on the image format. You can provide a larger value, in which case the extra bytes will extend beyond the end of each row of pixels. You may want to do so either to improve performance, or to describe an image within a larger image without copying the data. The extra bytes aren't considered part of the image represented by the vImage buffer. When allocating floating-point data for images, keep the data 4-byte aligned by allocating bytes as integer multiples of 4. For best performance, allocate bytes as integer multiples of 16.

    Look at the following image:

    enter image description here

    Red circle (top/left corner) is offset from the buffer start, let's calculate it (assuming 4 bytes per pixel):

    size_t startY = aspectRect.origin.y;
    size_t startX = aspectRect.origin.x;
    size_t offset = 4 * (finalWidth * startY + startX);
    

    The distance, in bytes, between the start of one pixel row and the next in an image, including any unused space between them is finalWidth * 4 (red line between two other circles).

    Let's fix the destBuffer:

    vImage_Buffer destBuffer = {
        (void *)destData+offset,
        aspectRect.size.height,
        aspectRect.size.width,
        finalWidth * 4
    };