Render video to specific region in a canvas

Question

I created my own canvas (the black area) on which I draw element that are not derived from Shape. Instead I draw using the DrawingContext in overriden method Canvas.OnRender.

The problem is, that I want to play a video file and render the frames into a specific region (for example the red rectangle) of my canvas - also in OnRender. But usually the MediaPlayer is directly bound to a brush which fills the complete background of an UIElement.

Any help?

Okay, here some more explanation. I can scroll, pan and zoom my canvas in any direction which then effects the content that I draw in OnRender using the drawingContext. The following picture shows a canvas with an image (the door) that comes frame by frame from a webcam. Therefore I can use the BitmapSource within drawingContext.DrawImage whenever my canvas is invalidated. And I can still draw my primitives. The problem with the MediaPlayer/MediaElement is that I don't get it frame by frame but still I want to render it similar to the webcam image within my canvas.

Answer 1

Finally I figured out how to achieve that. You simply need a MediaPlayer and feed it into drawingContext.DrawVideo in each Canvas.OnRender loop. Sample picture and code below. I also included methods to capture a frame (BitmapSource) from the video and also how to convert that into an old System.Drawing.Bitmap).

public partial class RenderCanvas : UserControl
{
    readonly MediaPlayer player;

    public RenderCanvas()
    {
        InitializeComponent();

        player = new MediaPlayer();
        player.Open(new Uri(@"test.avi", UriKind.Relative));
        player.Play();
    }

    protected override void OnRender(DrawingContext drawingContext)
    {
        base.OnRender(drawingContext);

        if (player != null && player.Source != null)
            drawingContext.DrawVideo(player, new Rect(0, 0, 200, 150));

        // draw any shape in front of the video
        drawingContext.DrawEllipse(Brushes.Blue, new Pen(Brushes.Red, 5), new Point(150, 150), 60, 60);
    }

    BitmapSource GetBitmapSourceFromVideo()
    {
        var drawingVisual = new DrawingVisual();
        var renderTargetBitmap = new RenderTargetBitmap(player.NaturalVideoWidth, player.NaturalVideoHeight, 96, 96, PixelFormats.Default);
        using (var drawingContext = drawingVisual.RenderOpen())
        {
            drawingContext.DrawVideo(player, new Rect(0, 0, player.NaturalVideoWidth, player.NaturalVideoHeight));
        }

        renderTargetBitmap.Render(drawingVisual);

        return renderTargetBitmap;
    }

    System.Drawing.Bitmap GetBitmapFromVideo()
    {
        BitmapSource bitmapSource = GetBitmapSourceFromVideo();

        var encoder = new PngBitmapEncoder();
        encoder.Frames.Add(BitmapFrame.Create(bitmapSource));
        using (var stream = new MemoryStream())
        {
            encoder.Save(stream);
            stream.Seek(0, SeekOrigin.Begin);

            return (System.Drawing.Bitmap)System.Drawing.Image.FromStream(stream);
        }
    }
}

Here the XAML code of the sample application. The XAML of the RenderCanvas has no changes.

<Window x:Class="CanvasTest_OnRender.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:CanvasTest_OnRender="clr-namespace:CanvasTest_OnRender" Title="MainWindow" Height="350" Width="525">
    <Grid>
        <CanvasTest_OnRender:RenderCanvas />
    </Grid>
</Window>