Search code examples
mp4h.264

Muxing MP4 with a profile that has multiple SPS/PPS

The ISO/IEC 14496-15 specification "Information technology — Coding of audio-visual objects — Part 15: Advanced Video Coding (AVC) file format" that describes the avc1 box used in MP4 files states that the avc1 box can contain multiple SPS and PPS entries for a single sample description (not to be confused with repeated SPS/PPS). And I am not sure how to mux such streams:

  • So far I've seen H.264 high profile streams with multiple PPS nal units for a single sample description. What confuses me is that since I've never personally seen a H.264 stream with multiple SPS units - is that even possible and what is the reasoning behind that?.
  • If that is possible - what happens to the AVCProfileIndication and AVCLevelIndication fields in the avc1 box? Should they be filled from the first SPS entry? If there are multiple SPS entires, I guess the profile indication should not differ, but what about level?
Solution

  • The sps and pps have ids encoded as a integer. VCL NALUs tell the decoder "to decode me you need data from a pps with id X". Hence theoretically each VCL NALU can reference a SPS/PPS with a different id. I've never seen this done in practice. But it is part of the h.264 spec, so the container supports it. And Yes, Just use the first SPS for the profile/constraints/level bytes.