Search code examples
directshowh.264gstreamerlive-streamingmpeg2-ts

Why do I get these artifacts when using Microsoft MPEG-TS Demultiplexer and H.264 decoder on a H.264 MPEG-TS stream?

I have written a live push filter that gets an mpeg-ts stream containing h.264 video and aac audio. I set up a directshow pipeline and configure the output pins. I can render the h.264 stream but I get artifacts in the rendering as can be seen from this screenshot when streaming from gstreamer using a videotestsrc and the "ball" pattern.the "ball" patternThis screenshot should contain only one white dot on a black background. The two additional are "leftovers" that appear when the animation plays.

If I stream MPEG-2 and change the pipeline accordingly, the pattern renders without error. I have tried to use settings described on msdn to configure the pin, both using H264 and AVC1 explicitly providing the sequence header and so on. I still get the same kind of artifacts.

One interesting thing is that the artifacts mostly appear at the same frequency as the I-Frames arrive, and if we only send I-Frames (key-int-max=1), the artifacts disappear completely.

Also, the errors seem to appear at the top half of the image when the I-Frame interval is 60, i.e. every 2 seconds. When we change to one I-Frame every second frame (key-int-max=2), the artifacts appear only in a narrow strip at the top of the image.

The following gstreamer pipeline produced the video stream:

videotestsrc live-source=true pattern=ball ! video/x-raw-yuv,format=(fourcc)I420,width=1366,height=768,framerate=30/1 ! timeoverlay halign=left valign=bottom shaded-background=true ! x264enc bitrate=4096 tune=zerolatency ! h264parse ! queue ! mux. audiotestsrc wave=ticks volume=0.2 ! voaacenc ! mux. mpegtsmux name=mux ! udpsink host=<ip> port=<port>

This is what the pipeline looks like: Pipeline

The configuration in this example is majortype = MEDIATYPE_Video, subtype = MEDIASUBTYPE_H264, formattype = FORMAT_MPEG2Video. No sequenceheader specifically provided, etc.

So the question is, are these kinds of artifacts symptoms of some common configuration problem?

Solution

  • It turns out that the MPEG-2 Demultiplexer is not designed to handle H.264 content. That is why these effects appear.