Search code examples
videovideo-streamingmp4h.264

segmentation of MP4 video over RTP

I'm trying to understand how a MP4 video is going to be segmented over RTP in online streaming (it's coded with the H.264 codec). There are some tools like FFMPEG or MP4box, or we can use the ISO format itself [moov, pair of moof and mdat] for streaming. What I want to know is how these tools will segment mdat part of video? Will they separate all the units (a group of video and audio frames) of the MP4 video and make them into one segment with related moov part or they do it in some other way?

Or do they use H264 streaming and NAL units for streaming?

I want to know how it works.

Solution

  • Typically, the video streaming flow goes like this:

    Capture Video--> Encode-->                                     -->Decode-->Display
                            Mux--> Transmit--> Receive-->De-Mux  
Capture Audio--> Encode-->                                     -->Decode-->Display

    So, basically in your case, ffmpeg takes NAL units from H.264 and muxes it with encoded audio bitstream and transmits together.