Search code examples
c++opencvraspberry-picamera

OpenCV reading too few pixels per row from camera

I am trying to fetch a video stream from a camera connected to the serial camera interface on my Raspberry pi 4. To read the video stream I'm using OpenCV and I have set the resolution to the maximum supported as listed by

pi@raspberrypi:~ $ ffmpeg -f v4l2 -list_formats all -i /dev/video0

The output from this can be seen here: https://i.sstatic.net/PYQJm.jpg

When scaling and showing the image using imshow it produces the strange output as seen here: enter image description here

I have tried setting different resolutions and for lower resolutions the "bug" disappears, but the image doesn't entirely fit in the window, meaning some pixels are left out. I have also tried changing pixel format, to both H264 and MJPEG but this only gives me a VIDIOC_STREAMON: Operation not permitted.

My code:

int testCamera(){
    VideoCapture stream = VideoCapture(0);
    if (!stream.isOpened()) {
        std::cout << "failed to start video stream" << std::endl;
        return -1;
    }
    stream.set(CAP_PROP_FRAME_WIDTH, 4056);
    stream.set(CAP_PROP_FRAME_HEIGHT, 3040);
    int frameWidth = stream.get(CAP_PROP_FRAME_WIDTH);
    int frameHeight = stream.get(CAP_PROP_FRAME_HEIGHT);
    std::cout<<frameWidth<<"x"<<frameHeight<<std::endl; //Prints 4056x3040
 
    Mat frame;
    Mat resized;
    while(1){
        stream >> frame;
 
        resize(frame, resized, Size(1280, 720));
        imshow("frame", resized);
        if (waitKey(10) == 27) break;
    }
}

Is this a hardware fault rather than a mismatch in OpenCV? Does anyone know how to fix this?

Solution

  • The Raspberry Pi camera has a basic block size of 32x16 which means all image sizes are padded up till the width is a multiple of 32 pixels and the height is a multiple of 16 pixels.

    In your case, 4056x3040 would become 4064x3040.

    That actually makes your camera 12MP, so you must have the newer Raspberry Pi High Quality Camera.

    Whilst I am certain it applies to the V1 and V2 camera, I am not 100% certain that the 32x16 rule applies to High Quality Camera, but it seems likely. I am happy to be corrected if anyone has better information/sources.

    Just for reference, the v2 Raspberry Pi Camera reports this:

    [video4linux2,v4l2 @ 0x10051c0] Raw       :     yuv420p :     Planar YUV 4:2:0 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x10051c0] Raw       :     yuyv422 :           YUYV 4:2:2 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x10051c0] Raw       :       rgb24 :     24-bit RGB 8-8-8 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x10051c0] Compressed:       mjpeg :            JFIF JPEG : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x10051c0] Compressed:        h264 :                H.264 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x10051c0] Compressed:       mjpeg :          Motion-JPEG : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x10051c0] Raw       : Unsupported :           YVYU 4:2:2 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x10051c0] Raw       : Unsupported :           VYUY 4:2:2 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x10051c0] Raw       :     uyvy422 :           UYVY 4:2:2 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x10051c0] Raw       :        nv12 :         Y/CbCr 4:2:0 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x10051c0] Raw       :       bgr24 :     24-bit BGR 8-8-8 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x10051c0] Raw       :     yuv420p :     Planar YVU 4:2:0 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x10051c0] Raw       : Unsupported :         Y/CrCb 4:2:0 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x10051c0] Raw       : Unsupported :  32-bit XBGR 8-8-8-8 : {32-3280, 2}x{32-2464, 2}