Play MPEG-2 TS using MseStreamSource

Question

I need to display a live video stream in a UWP application.

The video stream comes from a GoPro. It is transported by UDP messages. It is a MPEG-2 TS stream. I can play it successfully using FFPlay with the following command line :

ffplay -fflags nobuffer -f:v mpegts udp://:8554

I would like to play it with MediaPlayerElement without using a third party library.

According to the following page : https://learn.microsoft.com/en-us/windows/uwp/audio-video-camera/supported-codecs UWP should be able to play it. (I installed the "Microsoft DVD" application in the Windows Store).

I receive the MPEG-2 TS stream with a UdpClient. It works well. I receive in each UdpReceiveResult a 12 bytes header, followed by 4, 5, 6, or 7 MPEGTS packets (each packet is 188 bytes, beginning with 0x47).

I created a MseStreamSource :

_mseStreamSource = new MseStreamSource();
_mseStreamSource.Opened += (_, __) =>
{
    _mseSourceBuffer = _mseStreamSource.AddSourceBuffer("video/mp2t");
    _mseSourceBuffer.Mode = MseAppendMode.Sequence;
};
_mediaPlayerElement.MediaSource = MediaSource.CreateFromMseStreamSource(_mseStreamSource);

This is how I send the messages to the MseStreamSource :

    UdpReceiveResult receiveResult = await _udpClient.ReceiveAsync();
    byte[] bytes = receiveResult.Buffer;
    mseSourceBuffer.AppendBuffer(bytes.AsBuffer());

The MediaPlayerElement displays the message "video not supported or incorrect file name". (not sure of the message, my Windows is in French).

Is it a good idea to use the MseAppendMode.Sequence mode ? What should I pass to the AppendBuffer method ? The raw udp message including the 12 bytes header or each MPEGTS 188 bytes packet ?

Answer 1

I finally got the video working !

Here are the steps I follow to extract the MPEG-TS packets and correctly send them to the MseStreamSource :

The MseSourceBuffer needs to be in "Sequence" mode :

_mseSourceBuffer.Mode = MseAppendMode.Sequence;

For each received UDP datagram, I extract the MPEG-TS packets. To do that, I ignore the first 12 bytes of the UDP datagram. Then I extract each 188 bytes packet in a separate array (each packet starts with 0x47).

I send each packet to a synchronized queue.

I dequeue the packets from the queue and send them grouped to the MseSourceBuffer. I create a new group for each PAT packet (pid = 0) :

byte[] bytes;
// [...] combine the packets of the group
mseSourceBuffer.AppendBuffer(bytes.AsBuffer());

I tried to use a MemoryStream and call the AppendStream() method, but with no success.

Also care about threads synchronization : packets order should not be lost. That is the reason for the synchronized queue.

Hope it can help someone else.

This wikipedia MPEG-TS page helped me a lot.