Search code examples
videoffmpegframempegffprobe

ffmpeg - Retrieving video macroblock info with -debug mb_type command

I am trying to get the information about all the macroblocks in the frames of a video (mp4). In particular i'm using the ffmpeg command:

ffmpeg -debug mb_type -i input.mp4 out.mp4 2> macroblocks.txt

It seems to work fine, but... i don't understand how to parse the output!

I see that after many uninteresting writings, starts a group of blocks starting with

"New frame, type: [FRAME TYPE]"

so I assume that these are the blocks referring to each frame containing the type of each macroblock.. but what do the symbols inside mean?

New frame, type: B [h264 @ 000001c0241c1cc0] d < X- < I > > > > X d d d d d < < d < d > < d d > d < d d d < > < d < > X < d d > d X d < > d X d > > d d+ d

From the theory I know that there are intra or predicted macroblocks, but i don't understand how to parse this info from the "New frame"-blocks.

  • What means i,I,A,<,>,X,|,etc.?

Also often there are sentences like

nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2

or

cur_dts is invalid (this is harmless if it occurs once at the start per stream)

that i really don't understand... I can't too find a documentation.. Can anyone help me?

Solution

  • The best documentation seems to be the source code, currently starting at line 196 of libavcodec/mpegutils.c. I won't duplicate everything here, just enough to understand the example line given above.

    Each macroblock is described by 3 characters:

    1. type and motion vector direction

      • d: IS_DIRECT && IS_SKIP
      • <: !USES_LIST(0) - Reference to future (List 1, B slices)
      • X: USES_LIST(0) && USES_LIST(1) - Reference to past and future (List 1 & 2, B slices)
      • >: !USES_LIST(1) - Reference to past (List 0, P or B slices)
      • for more see the code

    2. segmentation

      • +: IS_8X8
      • -: IS_16X8
      • |: IS_8X16
      • space: IS_INTRA || IS_16X16
      • ?: otherwise

    3. interlacing

      • =: IS_INTERLACED
      • space: not interlaced

    Also interesting in this connection is the macro block type visualization built into ffmpeg itself.

    For the NAL unit types see table 7-1 here.