UWP: How to record Screen and save as mp4 file?

Question

I have figured out how to capture the screen and output it back to a previewer using the sample here: Microsoft Screen Capture Documentation

Which is fine as far as it goes.

What I can't figure out, nor can I find any documentation is to take those frames and write them to a video file.

Ideally I want to stream them directly into an mp4 or similar which I can then use later with the MediaComposition system to do editing.

I found VideoFrame.CreateWithDirect3D11Surface but I can't figure out how to add a video frame to an existing video file. The documentation tells you how to create a video frame and how the properties work, but it doesn't tell you how to use a video frame inside a video file, nor can I find anything about how to create a video file without a camera or other capture device.

There's also some reference from the documentation GitHub where someone is asking the same where they say MediaStreamSample is the key, but there isn't any code for that either and certainly nothing that allows saving of the file. (Here's the issue)

One would think that it would be easy to record the screen with this api and dump those frames to a raw video file that you could then bring in and edit using the MediaComposition apis.

Help please!

Answer 1

Assuming you are working from Microsoft's Screen Capture examples. Here's what you'll need to do:

Full source code is available here (The code does need refactoring but it works 🤷‍♂️): https://gitlab.com/colinkiama/screenrecordertest/-/tree/master/screenRecorderTest

The code is a uses the Screen Capture Example and parts of SimpleRecorder's source code: https://github.com/robmikh/SimpleRecorder/tree/master/SimpleRecorder

1. Create a MediaStreamSource with details about the input

private void CreateMediaObjects()
 {
            // Describe our input: uncompressed BGRA8 buffers
            var videoProperties = VideoEncodingProperties.CreateUncompressed(MediaEncodingSubtypes.Bgra8, _sourceWidth, _sourceHeight);
            _videoDescriptor = new VideoStreamDescriptor(videoProperties);


            // Create our MediaStreamSource
            _mediaStreamSource = new MediaStreamSource(_videoDescriptor);
            _mediaStreamSource.BufferTime = TimeSpan.FromSeconds(0);
            _mediaStreamSource.Starting += OnMediaStreamSourceStarting;
            _mediaStreamSource.SampleRequested += OnMediaStreamSourceSampleRequested;

            // Create our transcoder
            _transcoder = new MediaTranscoder();
            _transcoder.HardwareAccelerationEnabled = true;

        }

2. Set up the encoding, start capturing the screen and start transcoding (Note: stream refers to a stream opened from a StorageFile object)

private async Task EncodeInternalAsync(IRandomAccessStream stream, uint width, uint height, uint bitrateInBps, uint frameRate)
        {
            if (!_isRecording)
            {
                _isRecording = true;

                var encodingProfile = new MediaEncodingProfile();
                encodingProfile.Container.Subtype = "MPEG4";
                encodingProfile.Video.Subtype = "H264";
                encodingProfile.Video.Width = width;
                encodingProfile.Video.Height = height;
                encodingProfile.Video.Bitrate = bitrateInBps;
                encodingProfile.Video.FrameRate.Numerator = frameRate;
                encodingProfile.Video.FrameRate.Denominator = 1;
                encodingProfile.Video.PixelAspectRatio.Numerator = 1;
                encodingProfile.Video.PixelAspectRatio.Denominator = 1;

                StartFrameCapture();

                var transcode = await _transcoder.PrepareMediaStreamSourceTranscodeAsync(_mediaStreamSource, stream, encodingProfile);
                await transcode.TranscodeAsync();
            }
        }

3. For each frame that arrives from the screen capture, create a MediaStream Sample

using (var frame = _framePool.TryGetNextFrame())
                {
                    MediaStreamSample sampleToUseLater = MediaStreamSample.CreateFromDirect3D11Surface(frame.Surface, frame.SystemRelativeTime);

                    lock (doubleBufferingPool)
                    {
                        while (doubleBufferingPool.Count >= 2)
                        {
                            // Stops too many samples from being saved
                            doubleBufferingPool.Dequeue();
                        }

                        doubleBufferingPool.Enqueue(sampleToUseLater);
                    }
                }

4. Set the actual start time of the stream:

 private void OnMediaStreamSourceStarting(MediaStreamSource sender, MediaStreamSourceStartingEventArgs args)
        {

            while (doubleBufferingPool.Count == 0)
            {

            }
            var sample = doubleBufferingPool.Dequeue();
            TimeSpan timeStamp = sample.Timestamp;
            args.Request.SetActualStartPosition(timeStamp);
        }

5. Provide samples to the MediaStreamSource

private void OnMediaStreamSourceSampleRequested(MediaStreamSource sender, MediaStreamSourceSampleRequestedEventArgs args)
        {
            if (_isRecording && !_closed)
            {
                while (doubleBufferingPool.Count == 0)
                {

                }

                lock (doubleBufferingPool)
                {
                    args.Request.Sample = doubleBufferingPool.Dequeue();
                }
            }
            else
            {
                args.Request.Sample = null;
                StopCapture();
            }
        }

If you provide the MediaStreamSource with a null sample, it will stop asking for more samples.

The video will then be available to view from the file you opened the stream from.