Search code examples
ffmpeg

Is extract_mvs in ffmpeg able to extract mvs of sub-macroblocks(8x4/4x8, 4x4)?


I want to get information of motion vectors(mvs) of h264 video, and I have tried to use ffmpeg's extract_mvs.c to slove this problem.

However, I found that the extracted mvs's block size are 16x16, 16x8, 8x16 or 8x8, but none of them is 8x4, 4x8 or 4x4.

I wonder if there is any way to get the sub-macroblock's mvs?

I think there do exist mvs of sub-macroblocks(8x4, 4x8 or 4x4), as I can get information(use x264.exe) like this:

x264 [info]: mb I  I16..4: 39.9% 24.5% 35.6%
x264 [info]: mb P  I16..4:  1.9%  0.4%  0.2%  P16..4: 20.1% 17.9% 28.4% 11.2%  7.2%    skip:12.7%
x264 [info]: mb B  I16..4:  0.0%  0.0%  0.0%  B16..8: 42.3%  6.6%  1.8%  direct: 2.3%  skip:47.0%  L0:31.8% L1:44.3% BI:24.0%

'P16..4: 20.1% 17.9% 28.4% 11.2% 7.2%' is the rate of 16x16, 16x8/8x16, 8x8, 8x4/4x8, and 4x4.
But I don't know how to get mvs of sub-macroblocks(8x4, 4x8 or 4x4).


Solution

  • You can tell the FFmpeg H264 decoder to export MVs using -flags2 +export_mvs (before -i). This should export MVs as frame side-data of type AV_FRAME_DATA_MOTION_VECTORS using the data structures in motion_vector.h. An example consumer of this side-data is the codecview video filter, which will draw the MVs on the frame itself. But you can do whatever you want with that side-data, it's just extra information on top of the frame itself. And the header suggests all blocksizes are supported, not just 16x16 MBs.

    [edit] see comment below, it appears this is unfortunately not possible for sub8x8 block sizes.