Getting System.ExecutionEngineException in procedure when resizing window quickly

Question

When I resize my WinUI 3 app, it has an System.ExecutionEngineException in the procedure I have to detect whenever a hotkey is pressed.

System.ExecutionEngineException HResult=0x80131506 Source= StackTrace:

return PInvoke.CallWindowProc(wndProc, hWnd, Msg, wParam, lParam);

In my app, I have a ToggleButton that I want to be toggled by the F6 key. Everything works fine in terms of key properly toggling the button, but whenever I quickly resize the app it crashes.

I have tried to check whether the window handle is null but that didn't solve it. I also tried to get a better error message by handling it but was unsuccessful.

Reproduction

GitHub Repository

I am using Visual Studio 2022 with the Blank App, Packaged (WinUI 3 in Desktop) with the CsWin32 package installed.

MainWindow.xaml.cs:

using System;
using System.Runtime.InteropServices;
using Microsoft.UI.Xaml;
using Windows.Win32;
using Windows.Win32.Foundation;
using Windows.Win32.UI.Input.KeyboardAndMouse;
using Windows.Win32.UI.WindowsAndMessaging;

namespace HotkeySubclassing;

public sealed partial class MainWindow : Window
{
    private WNDPROC wndProc = null!;

    private LRESULT HotKeyProc(HWND hWnd, uint Msg, WPARAM wParam, LPARAM lParam)
    {
        uint WM_HOTKEY = 0x0312; // HotKey Window Message

        if (Msg == WM_HOTKEY)
        {
            if (ToggleButton.IsEnabled)
            {
                ToggleButton.IsChecked = !ToggleButton.IsChecked;
            }
        }

        return PInvoke.CallWindowProc(wndProc, hWnd, Msg, wParam, lParam); // Error happens on this line
    }

    public MainWindow()
    {
        this.InitializeComponent();

        // Get window handle
        HWND hWnd = new(WinRT.Interop.WindowNative.GetWindowHandle(this));

        // Register hotkey
        int id = 0x0000;
        _ = PInvoke.RegisterHotKey(hWnd, id, HOT_KEY_MODIFIERS.MOD_NOREPEAT, 0x75); // F6

        // Add hotkey function pointer to window procedure
        WNDPROC hotKeyDelegate = HotKeyProc;
        nint hotKeyProcPtr = Marshal.GetFunctionPointerForDelegate(hotKeyDelegate);
        nint wndPtr = PInvoke.SetWindowLongPtr(hWnd, WINDOW_LONG_PTR_INDEX.GWL_WNDPROC, hotKeyProcPtr);
        wndProc = Marshal.GetDelegateForFunctionPointer<WNDPROC>(wndPtr);
    }
}

NativeMethods.txt:

RegisterHotKey
UnregisterHotKey
SetWindowLongPtr
CallWindowProc

MainWindow.xaml:

<?xml version="1.0" encoding="utf-8" ?>
<Window
    x:Class="HotkeySubclassing.MainWindow"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    Title="HotkeySubclassing"
    mc:Ignorable="d">

    <StackPanel
        HorizontalAlignment="Center"
        VerticalAlignment="Center"
        Orientation="Horizontal">
        <ToggleButton x:Name="ToggleButton">Click Me</ToggleButton>
    </StackPanel>
</Window>

Answer 1

It's because when running .NET you're living in a "managed" world, and objects can be moved in memory when .NET needs to, typically when the garbage collector runs. So what happens is you pass a pointer to WinUI3 code and that pointer goes rogue at some moment and you get a big crash.

So you must "pin" the objects you pass to unmanaged memory so they won't be moved. The easiest way is to create a static variable, something like this:

public sealed partial class MainWindow : Window
{
    private static WNDPROC? wndProc;
    private static WNDPROC hp = HotKeyProc; // fixes method pointer
    private static MainWindow? _window;

    private static LRESULT HotKeyProc(HWND hWnd, uint Msg, WPARAM wParam, LPARAM lParam)
    {
        const uint WM_HOTKEY = 0x0312; // HotKey Window Message

        if (Msg == WM_HOTKEY)
        {
            if (_window?.ToggleButton.IsEnabled == true)
            {
                _window.ToggleButton.IsChecked = !_window.ToggleButton.IsChecked;
            }
        }

        return PInvoke.CallWindowProc(wndProc, hWnd, Msg, wParam, lParam);
    }

    public MainWindow()
    {
        this.InitializeComponent();

        _window = this;

        // Get window handle
        HWND hWnd = new(WinRT.Interop.WindowNative.GetWindowHandle(this));

        // Register hotkey
        int id = 0x0000;
        _ = PInvoke.RegisterHotKey(hWnd, id, HOT_KEY_MODIFIERS.MOD_NOREPEAT, 0x75); // F6

        // Add hotkey function pointer to window procedure
        nint hotKeyProcPtr = Marshal.GetFunctionPointerForDelegate(hp);
        nint wndPtr = PInvoke.SetWindowLongPtr(hWnd, WINDOW_LONG_PTR_INDEX.GWL_WNDPROC, hotKeyProcPtr);
        wndProc = Marshal.GetDelegateForFunctionPointer<WNDPROC>(wndPtr);
    }
}

There is a example here How to make a global keyboard accelerator/hotkey for a button? that has some reusable code and works a bit differently