How do I convert from mouse coordinates to pixel coordinates of a TransformedBitmap?

Question

My question is similar to How to get correct position of pixel from mouse coordinates?, with the added caveat that the image is potentially a TransformedBitmap, where flips and rotations could be applied, and still return the pixel coordinates of the original image.

My Window's design looks like this:

    <DockPanel>
        <Label DockPanel.Dock="Bottom" Name="TheLabel" />
        <Image DockPanel.Dock="Top" Name="TheImage" Stretch="Uniform" RenderOptions.BitmapScalingMode="NearestNeighbor" MouseMove="TheImage_MouseMove" />
    </DockPanel>

and the codebehind looks like this:

        public MainWindow()
        {
            InitializeComponent();

            const int WIDTH = 4;
            const int HEIGHT = 3;
            byte[] pixels = new byte[WIDTH * HEIGHT * 3];
            pixels[0] = Colors.Red.B;
            pixels[1] = Colors.Red.G;
            pixels[2] = Colors.Red.R;
            pixels[(WIDTH * (HEIGHT - 1) + (WIDTH - 1)) * 3 + 0] = Colors.Blue.B;
            pixels[(WIDTH * (HEIGHT - 1) + (WIDTH - 1)) * 3 + 1] = Colors.Blue.G;
            pixels[(WIDTH * (HEIGHT - 1) + (WIDTH - 1)) * 3 + 2] = Colors.Blue.R;
            BitmapSource bs = BitmapSource.Create(WIDTH, HEIGHT, 96.0, 96.0, PixelFormats.Bgr24, null, pixels, WIDTH * 3);
            TheImage.Source = bs;
        }

        private void TheImage_MouseMove(object sender, MouseEventArgs e)
        {
            Point p = e.GetPosition(TheImage);
            if (TheImage.Source is TransformedBitmap tb)
                TheLabel.Content = tb.Transform.Inverse.Transform(new Point(p.X * tb.PixelWidth / TheImage.ActualWidth, p.Y * tb.PixelHeight / TheImage.ActualHeight)).ToString();
            else if (TheImage.Source is BitmapSource bs)
                TheLabel.Content = new Point(p.X * bs.PixelWidth / TheImage.ActualWidth, p.Y * bs.PixelHeight / TheImage.ActualHeight).ToString();
        }

When hovering in the bottom-right corner (which I colored blue for easy tracking) of the non-transformed image, you correctly see the coordinates of (~4, ~3), which are the image dimensions.

However, once you apply a transform, for example changing TheImage.Source = bs; to TheImage.Source = new TransformedBitmap(bs, new RotateTransform(90.0));, hovering over the blue instead gives (~4, ~0).

I think one could look at the actual matrix values of the transform, and determine how to adjust the point in all the various cases, but it seems like there should be an easier solution using the inverse transform.

Answer 1

When an Image element renders a TransformedBitmap, the center point of the Transform is effectively ignored and the bitmap is shifted into an appropriate coordinate range.

Since this shift is not applied to the transfomation of a Point, you have to compensate it, e.g. like this:

var p = e.GetPosition(TheImage);

if (TheImage.Source is BitmapSource bs)
{
    p = new Point(
        p.X * bs.PixelWidth / TheImage.ActualWidth,
        p.Y * bs.PixelHeight / TheImage.ActualHeight);

    if (TheImage.Source is TransformedBitmap tb)
    {
        var w = tb.Source.PixelWidth;
        var h = tb.Source.PixelHeight;

        p = tb.Transform.Inverse.Transform(p);

        p = new Point((p.X + w) % w, (p.Y + h) % h);
    }

    TheLabel.Content = $"x: {(int)p.X}, y: {(int)p.Y}";
}

The above code will however not work correctly when the Transform center is set to a value other than (0,0).

In order to make it work for Transforms with arbitrary center point, you have to clear the offset values in the transform matrix:

var p = e.GetPosition(TheImage);

if (TheImage.Source is BitmapSource bs)
{
    p = new Point(
        p.X * bs.PixelWidth / TheImage.ActualWidth,
        p.Y * bs.PixelHeight / TheImage.ActualHeight);

    if (TheImage.Source is TransformedBitmap tb)
    {
        var w = tb.Source.PixelWidth;
        var h = tb.Source.PixelHeight;
        var t = tb.Transform.Value;
        t.Invert();
        t.OffsetX = 0d;
        t.OffsetY = 0d;

        p = t.Transform(p);

        p = new Point((p.X + w) % w, (p.Y + h) % h);
    }

    TheLabel.Content = $"x: {(int)p.X}, y: {(int)p.Y}";
}