Search code examples
ffmpegjpegcolor-space

How to preserve color when extracting thumbnail from videos with FFMPEG?

I'm using FFMPEG to extract thumbnails from arbitrary user-submitted videos to jpg and the outputs are noticeably darker than the input video.

Here is an image captured of a MP4 input (SMPTE Color Bars) using Mac OS's built in screen capture utility (which seems to have a reasonably accurate color representation. Red value on this image is R: 191, G: 0, B: 1.
brighter image of SMPTE color bars

Here is the same image captured using the FFMPEG script below. Red value on this image is R: 176, G: 0, B: 2.
diminished brightness image of SMPTE color bars

Here is the ffmpeg options I'm using to extract these images:

ffmpeg -i input-video.mp4 \
    -vframes 1 \
    -filter:v scale=600:-1 \
    -qscale:v 90 \
    -f singlejpeg \
    thumb.jpg

Any thoughts on how to solve this?

(Note that while there are related topics on Stack Overflow, all of them are specific to known input video types. I'm looking for a solution that will work across arbitrary inputs.)

Solution

  • There is a bug in FFmpeg that also applies the latest stable current release (version 5.1.2).
    I was able to reproduce the issue using Windows 10, so the bug is not specific to MacOS.

    [Here is a question from 2016, describing the same issue].

    The solution I found is appending out_color_matrix=bt601:out_range=pc to the scale filter:

    ffmpeg -y -i color_bars.mp4 -vframes 1 -filter:v scale=600:-1:out_color_matrix=bt601:out_range=pc -qscale:v 0 thumb.jpg

    • out_color_matrix=bt601 - applies BT.601 as output color space.
    • out_range=pc - applies full range of YUV (full range is [0, 255], and in "TV range" / "limited range", the range of Y is [16, 235]).

    According Wikipedia JPEG color conversion applies BT.601 "full range" YUV format, and the arguments out_color_matrix=bt601:out_range=pc were selected accordingly.

    Adding out_color_matrix=bt601:out_range=pc, forces FFmpeg to apply color format conversion that matches JPEG standard.

    The bug is that by default FFmpeg doesn't apply color format conversion, and keeps the original color format, which is presumably "limited range" BT.709. Keeping the original color format (without conversion) results a "color shift".

    Reproducing the issue:

    • Download the PNG image from the question and rename the file to color_bars.png.
    • Create an MP4 video from the PNG image:
      ffmpeg -y -i color_bars.png -loop 100 -vcodec libx264 -vf scale=out_color_matrix=bt709:out_range=tv -crf 10 -pix_fmt yuv444p -bsf:v h264_metadata=video_full_range_flag=0:colour_primaries=1:transfer_characteristics=1:matrix_coefficients=1 color_bars.mp4
      The conversion applies BT.709 "limited range" format.
      The bitstream filter marks the H.264 video stream as BT.709 "limited range" format.
    • Convert single frame to JPEG without color conversion:
      ffmpeg -y -i color_bars.mp4 -vframes 1 -filter:v scale=600:-1 -qscale:v 90 thumb.jpg
    • Convert single frame to JPEG with color conversion:
      ffmpeg -y -i color_bars.mp4 -vframes 1 -filter:v scale=600:-1:out_color_matrix=bt601:out_range=pc -qscale:v 0 thumb2.jpg

    Results:
    thumb.jpg - without color conversion:
    enter image description here
    The RGB values of the red pixels are [176, 0, 2].

    thumb2.jpg - with color conversion:
    enter image description here
    The RGB values of the red pixels are [190, 0, 0] (there is an acceptable rounding error).