Search code examples
parsingffmpegmp4mpeg-4

Why can't I get a manually modified MPEG-4 extended box (chunk) size to work?

Overview

As part of a project to write an MPEG-4 (MP4) file parser, I need to understand how an extended box (or chunk) size is processed within an MP4 file. When I tried to manually simulate an MP4 file with an extended box size, media players report that the file is invalid.

Technical Information

Paraphrasing the MPEG-4 specification:

An MP4 file is formed as a series of objects called 'boxes'. All data is contained in boxes, there is no other data within the file.

Here is a screen capture of Section 4.2: Object Structure, which describes the box header and its size and type fields:

MPEG-4 Object Structure (part 1)

enter image description here

Most MP4 box headers contain two fields: a 32-bit compact box size and a 32-bit box type. The compact box size supports a box's data up to 4 GB. Occasionally an MP4 box may have more data than that (e.g., a large video file). In this case, the compact box size is set to 1, and eight (8) octets are added immediately following the box type. This 64-bit number is known as the 'extended box size', and supports a box's size up to 2^64.

To understand the extended box size better, I took a simple MP4 file and wanted to modify the moov/trak/mdia box to use the extended box size, rather than the compact size.

Here is what the MP4 file looks like before modifying it. The three box headers are highlighted in RED:

MP4 file before inserting an extended box size

My plan was as follows:

  1. Modify the moov/trak/mdia box
    • In the moov/trak/mdia, insert eight (8) octets immediately following the box type ('mdia'). This will eventually be our extended box size.
    • Copy the compact box size to the newly-inserted extended box size, adding 8 to the size to compensate for the newly inserted octets. The size is inserted in big-endian order.
    • Set the compact size to 1.
  2. Modify the moov/trak box
    • Add 8 to the existing compact box size (to compensate for the eight octets added to mdia).
  3. Modify the moov box
    • Add 8 to the existing compact box size (again, to compensate for the eight octets in mdia)

Here's what the MP4 file looks like now, with the modified octets are in RED:

MP4 file after inserting an extended box size

What have we done?

We have told the MP4 parser/player to take the moov/trak/mdia box size from the extended field rather than the compact size field, and have increased all parent boxes by eight (8) to compensate for the newly-inserted extended box size in the mdia box.

What's the problem?

When I attempt to play the modified MP4 file I receive error messages from different media players:

Windows Media Player

Windows Movies & TV App

Why do the media players see the modified file as invalid MP4?

  • Did I need to alter any other fields?
  • Does the extended box size have to be greater than 2^32?
  • Can it be that only specific box types support extended box size (e.g., Media Data)?
Solution

  • A tip of the hat to @Alan Birtles for pointing out that the chunk offsets would also need to be modified. Indeed, the stco (sample table chunk offset?) box contains absolute file offsets to the data chunks in the mdat box (rather than relative offsets within a box). This can be seen in the specification document:

    MPEG-4 Specification for stco box

    The chunk offsets need to be increased by the number of octets we added to the file before the mdat box. In our case, this is the eight (8) octet extended box size inserted in the mdia box.

    All that remained was to manually change the chunk offsets found in the two stco boxes (both video and audio tracks), adding eight (8) to each chunk offset. Here are the stco boxes before adding 8 to their chunk offsets:

    stco box: audio stco box: video

    Now the file passes validity tests of both the ffmpeg and ffprobe tools. Interestingly, although VLC succeeds in playing the modified file, other media players (e.g., Windows Media Player, MS Photos, MS Movies & TV, MS MovieMaker) report the file as corrupted. It is not clear why they fail to play the file. Unverified possibilities include:

    • Not supporting the extended box size for any box other than mdat
    • Balking if the extended box size is less than 2^32

    In summary, if any fields are added to boxes (e.g., extended box size), the stco chunk offsets need to be incremented by the number of octets inserted in the MP4 file preceding each stco box.