What are these extra bytes coming from the iPhone camera in portrait mode?

Question

When I get a frame from - (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection I am getting back the following data:

• BytesPerRow: 1,472 Length: 706,560 Height: 480 Width: 360 format: BGRA

This is from the front camera on an iPhone 6 plus. This doesn't make sense because bytes per row should be (width * channels) (channels in this case is 4). However, it's (width+8)*channels. Where is this extra 8 bytes coming from?

Here's my code: Attaching the output to the session I set the orientation to portrait

bool attachOutputToSession(AVCaptureSession *session, id cameraDelegate)
{
    assert(cameraDelegate);

    AVCaptureVideoDataOutput *m_videoOutput = [[AVCaptureVideoDataOutput alloc] init];

    //create a queue for capturing frames
    dispatch_queue_t captureQueue  = dispatch_queue_create("captureQueue", DISPATCH_QUEUE_SERIAL);

    //Use the AVCaptureVideoDataOutputSampleBufferDelegate capabilities of CameraDelegate:
    [m_videoOutput setSampleBufferDelegate:cameraDelegate queue:captureQueue];

    //setup the video outputs
    m_videoOutput.alwaysDiscardsLateVideoFrames = YES;

    NSNumber *framePixelFormat = [NSNumber numberWithInt:kCVPixelFormatType_32BGRA];//This crashes with 24RGB b/c that isn't supported on iPhone
    m_videoOutput.videoSettings = [ NSDictionary dictionaryWithObject:framePixelFormat forKey:(id)kCVPixelBufferPixelFormatTypeKey];

    //Check if it already has an output from a previous session
    if ([session canAddOutput:m_videoOutput])
    {
        [session addOutput:m_videoOutput];
    }

    //set connection settings
    for (AVCaptureConnection *connection in m_videoOutput.connections)
    {
        if (connection.isVideoMirroringSupported)
            connection.videoMirrored = true;
        if (connection.isVideoOrientationSupported)
            connection.videoOrientation = AVCaptureVideoOrientationPortrait; 
    }

    return true;
}

When I set the orientation to LandscapeRight I do not have this issue. The bytes per row is equal to width*channels.

Here's where I'm getting the numbers mentioned above:

-(void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection
{
    CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);

    CIImage *ciImage = [CIImage imageWithCVPixelBuffer:imageBuffer];

    CVPixelBufferLockBaseAddress(imageBuffer,0);
    size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);
    size_t width = CVPixelBufferGetWidth(imageBuffer);
    size_t height = CVPixelBufferGetHeight(imageBuffer);
}

Answer 1

OK turns out this is part of the image "stride". If an image width is not divisible by the chosen memory allotment then this extra padding is included. When I receive the portrait image it is 360x480. Since 360 is not divisible by 16, 8 extra bytes are added as padding. 16 is the memory space in this case. I was not having this issue before because 480 is divisible by 16. You can get this number by calling CVPixelBufferGetBytesPerRowOfPlane (imageBuffer, 1); What's weird though, is that it returns a 0 the first time, 1 the second time, and so on until it reaches the real buffer level (8). Then it returns 0 again on the ninth image.

According to rpappalax on this page http://gstreamer-devel.966125.n4.nabble.com/iOS-capture-problem-td4656685.html

The stride is effectively CVPixelBufferGetBytesPerRowOfPlane() and includes padding (if any). When no padding is present CVPixelBufferGetBytesPerRowOfPlane() will be equal to CVPixelBufferGetWidth(), otherwise it'll be greater.

Although that wasn't exactly my experience.