10 bit Video Processing in Python with OpenCV

Question

I need to process a video with bit depth of 10. I want to keep all the information but OpenCV keeps converting images to 8 bit. Is there a way to keep the bit depth while processing images?

My code is as following:

import cv2
import numpy as np

cv2.namedWindow("ROI", cv2.WINDOW_NORMAL)
video_capture = cv2.VideoCapture("Ciftci.MP4",)
video_capture.set(cv2.CAP_PROP_CONVERT_RGB, 0)
ret, frame0 = video_capture.read()
column, row, height, width = cv2.selectROI('ROI', frame0)

But I keep receiving following error:

[ WARN:[email protected]] global cap_ffmpeg_impl.hpp:1592 retrieveFrame Unknown/unsupported picture format: yuv422p10le, will be treated as 8UC1.

Thanks in advance

Answer 1

It might be that the default OpenCV install with default build options/ffmpeg doesn't have support for this format / codec (e.g. HVEC iOS videos, etc.).

Doing a quick search I spotted this ffmpeg Chroma Subsampling setting. I haven't tested this yet, but I suspect it should be possible to:

1. grab a version of ffmpeg with this feature available
2. build OpenCV from source with this version of ffmpeg and Python bindings so hopefully cv2.VideoCapture will have yuv422p10le support.

Setting up/building OpenCV and its many dependencies, depending on your OS can take a bit of time/effort so here's another idea (with two varations): prototype your idea with 10bit data before committing the time to a custom OpenCV Python build.

The idea is to convert your video to an image sequence first (might be 16bit PNG, you can use ffmpeg or other tools). You can try to load the sequence via cv2.VideoCapture by also using the cv2.CAP_IMAGES flag (though I can't remember if it can handle 16bit images by default). (Bear in mind the formatted image sequence name (e.g. img_%03d.png (for something like img_001.png...). If your image sequence is named properly even providing the filename of the first frame might work (e.g. cv2.VideoCapture("frames/img_001.png", cv2.CAP_IMAGES)).

If cv2.VideoCapture doesn't handle 16bit images you should still be able to hold a reference to a Mat you load via cv2.imread() with the cv2.IMREAD_ANYDEPTH flag)

(For visualisation purposes, if using a depth map remember to normalise or rescale a Mat before cv2.imshow to preview in a helpful way.)