Search code examples
winapiencodingdirectshowvideo-capturedirectshow.net

DirectShow Encoding dropped frames compensation

I currently have a DirectShow graph in GraphEdit which looks like this:

      Source(USB WebCam) -> Encoder -> Mux -> File Writer

However, if the Source drops frames under recording, the encoder won't compensate for that loss.. Meaning if i have recorded 30 seconds of video, and dropped 20% of the frames (as an example) the resulting video would end with 24 seconds of video.

This means that everything in the video would be sped up by a factor of 1.2x which is absolutely not acceptable, I've replayed the video with VLC Player and WMP - same results. It's as if the encoder(?) or some other filter in the graph does not care about the lost frames.

Encoders I've used to reproduce this problem is WMV9 and a third party H264 Filter.

Even though the CPU usage is very high during recording, I really need to be able to handle and compensate for dropped frames so that the timeline of the video doesn't suddenly stop making sense. Is there a filter out there that handles this, or do I need to do something myself in the graph? What's actually happening here?

Thanks a lot.

Edit: I have tried to build a much simpler graph using Asf Writer filter and also using an Audio Source as you can see in the attached photo. However because my Capture Source (Camera) loses frames, the end video ends up with a big sync issue meaning the audio length is way longer than the video and goes out of sync (instead of compensating) I Would have expected to see the video catching up to the audio.

DirectShow Simple Graph

Solution

  • If you end up with a file of smaller duration (by the length of dropped frames), the problem is a likely bug in one of the filters in the pipeline.

    Potential problems that might cause this behavior fall into two categories:

    1. Multiplexer ignores dropped frames and contracts resulting file
    2. One of the filters upstream to multiplexer is removing timestamps from the frames, which makes it impossible to "see" dropped frame and filters fall back (some multiplexers would abort with error instead) into assuming continuous video sequence
      • the problem might also take place if you use preview output of web camera instead of capture output