Search code examples
ffmpeg

ffmpeg, histeq, palettegen and paletteuse behaviour

Using ffmpeg, passing a video though histeq then palettegen and paletteuse generates strange output.

Example 1

wget https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/1080/Big_Buck_Bunny_1080_10s_2MB.mp4
 
ffmpeg -ss  00:00:09.126 -t 0:0:01.0 -i Big_Buck_Bunny_1080_10s_2MB.mp4  -filter_complex "histeq=0.01,split[a][b];[a] palettegen [p];[b][p] paletteuse" -y example.gif

example 1

Example 2

Adding another filter to the chain fixes it.

ffmpeg -ss  00:00:09.126 -t 0:0:01.0 -i Big_Buck_Bunny_1080_10s_2MB.mp4  -filter_complex "histeq=0.01,eq,split[a][b];[a] palettegen [p];[b][p] paletteuse" -y example.gif

example2

Note the additional eq filter after histeq in the second example but why should it make a difference? Shouldn't the first example give the same result?

UPDATE

The only difference between the two examples is the filter chain. So specifically my question is why do the following two filter chains produce different results?

# from example 1:
histeq=0.01,split[a][b];[a] palettegen [p];[b][p] paletteuse

# from example 2:
histeq=0.01,eq,split[a][b];[a] palettegen [p];[b][p] paletteuse
Solution

  • It looks like the input format to histeq is bgra applying histogram equalization to the alpha (transparency) channel gives strange results.

    For fixing the issue we have to explicitly select rgb24 (or bgr24) format before applying histeq filter:

    ffmpeg -ss 00:00:09.126 -t 0:0:01.0 -i Big_Buck_Bunny_1080_10s_2MB.mp4 -filter_complex "format=rgb24,histeq=0.01,split[a][b];[a]palettegen[p];[b][p]paletteuse" -y example.gif

    (For testing we may select format=bgra instead of format=rgb24, and get the poor output).

    histeq filter include automatic format conversion at the input, but due to a bug (I assume) the above filter chain causes it to pick the wrong format.

    Using -loglevel debug, we can see more details:
    ffmpeg -i Big_Buck_Bunny_1080_10s_2MB.mp4 -filter_complex "histeq=0.01,split[a][b];[a]palettegen[p];[b][p]paletteuse" -y example.gif

    Here we can see that filter 'auto_scale' is inserted before histeq filter, and applied conversion from yuv420p to bgra:

    [auto_scale_0 @ 00000255ecbe6440] w:iw h:ih flags:'' interl:0 [Parsed_histeq_0 @ 00000255eae16f00] auto-inserting filter 'auto_scale_0' between the filter 'graph 0 input from stream 0:0' and the filter 'Parsed_histeq_0' [AVFilterGraph @ 00000255ec41d080] query_formats: 7 queried, 7 merged, 1 already done, 0 delayed [auto_scale_0 @ 00000255ecbe6440] w:1920 h:1080 fmt:yuv420p sar:1/1 -> w:1920 h:1080 fmt:bgra sar:1/1 flags:0x0

    When using MP4 as output (without paltetgen): ffmpeg -loglevel debug -i Big_Buck_Bunny_1080_10s_2MB.mp4 -filter_complex "histeq=0.01" -y example.mp4, rgb24 format is picked (and the result look fine):

    [auto_scale_0 @ 000001d884baec80] picking rgb24 out of 6 ref:yuv420p alpha:0 [auto_scale_1 @ 000001d884baef80] picking yuv444p out of 13 ref:rgb24 alpha:0 [auto_scale_0 @ 000001d884baec80] w:1920 h:1080 fmt:yuv420p sar:1/1 -> w:1920 h:1080 fmt:rgb24 sar:1/1 flags:0x0 [auto_scale_1 @ 000001d884baef80] w:1920 h:1080 fmt:rgb24 sar:1/1 -> w:1920 h:1080 fmt:yuv444p sar:1/1 flags:0x0

    When eq filter is used before histeq, the input to eq is bgra.
    It's difficult to answer why eq fixes the problem (I assume that the output of eq in not bgra).

    Note:
    Applying histogram equalization on RGB is fine but not ideal (assume histogram equalization is applied on red, green and blue separately).
    When applying histogram equalization on colored image, it is recommended to convert the image to LAB color space, applying histogram equalization only on the luminance, and converting back to RGB.
    We can't apply the suggested procedure because FFmpeg doesn't support LAB color space.