Search code examples
video-streamingh.264rtp

How does a decoder determines the size of Single NAL Unit in H264 decoding

As mentioned in RFC 6184 https://www.rfc-editor.org/rfc/rfc6184#section-5.6 section 5.6 that Single NAL Unit packet can only contain one NAL unit.

My question is how does an decoder now at the receiving end identifies the size of this Single NAL unit or knows end of NAL unit in the Single NAL unit packet.

However in other packetization mode like STAP and others the size of a NAL unit is present as a part of the RTP payload.

Solution

  • My question is how does an decoder now at the receiving end identifies the size of this Single NAL unit or knows end of NAL unit in the Single NAL unit packet.

    The OS/socket API tells you what the size of a received UDP (RTP) packet is. In the case of TCP streaming, the size of the RTP packet is typically prepended to the RTP packet (both in RTSP and RFC4571). After processing the RTP header, the payload is the NAL unit in single nal unit mode.

    While a typical RTP header is 12 bytes, you should parse it according to RFC3550 as the size depends on CSRCs and RTP header extensions.

    In the case of STAP, you need to know the size as there are multiple NALUs in a single RTP packet. Hence you have to parse each one by reading the size.