Migrating code to C#/Win32 causes exceptions

Question

In our (WinUI3-based) project, we've been using the P/Invoke packages to call low-level methods that modify the windows. But recently all these packages have been deprecated in favor of the source-generated C#/win32 package.

However, I'm having some difficulties migrating all of our code to this new library. The biggest issue I'm having is with the win32 sub-classing. In our current code we have this working like this (written by a collegue that left the company a year ago):

private delegate IntPtr WinProc(IntPtr hWnd, PInvoke.User32.WindowMessage Msg, IntPtr wParam, IntPtr lParam);

private IntPtr _oldWndProc = IntPtr.Zero;

[DllImport("user32.dll", EntryPoint = "SetWindowLong")]
private static extern IntPtr SetWindowLongPtr32(IntPtr hWnd, PInvoke.User32.WindowLongIndexFlags nIndex, WinProc newProc);

[DllImport("user32.dll", EntryPoint = "SetWindowLongPtr")]
private static extern IntPtr SetWindowLongPtr64(IntPtr hWnd, PInvoke.User32.WindowLongIndexFlags nIndex, WinProc newProc);

// This static method is required because Win32 does not support GetWindowLongPtr directly
private static IntPtr SetWindowLongPtr(IntPtr hWnd, PInvoke.User32.WindowLongIndexFlags nIndex, WinProc newProc)
{
    if (IntPtr.Size == 8)
        return SetWindowLongPtr64(hWnd, nIndex, newProc);
    else
        return SetWindowLongPtr32(hWnd, nIndex, newProc);
}

[DllImport("user32.dll")]
static extern IntPtr CallWindowProc(IntPtr lpPrevWndFunc, IntPtr hWnd, PInvoke.User32.WindowMessage Msg, IntPtr wParam, IntPtr lParam);

private void SubClassingWin32()
{
    _hwnd = WinRT.Interop.WindowNative.GetWindowHandle(this);

    var newWndProc = new WinProc(NewWindowProc);
    _oldWndProc = SetWindowLongPtr(_hwnd, PInvoke.User32.WindowLongIndexFlags.GWL_WNDPROC, newWndProc);
}

private IntPtr NewWindowProc(IntPtr hWnd, PInvoke.User32.WindowMessage Msg, IntPtr wParam, IntPtr lParam)
{
    switch (Msg)
    {
        [...]
    }
    return CallWindowProc(_oldWndProc, hWnd, Msg, wParam, lParam);
}

I tried to rewrite that to C#/Win32, but I get Execution Engine Exceptions.

private delegate LRESULT WNDPROCDelegate(HWND hWnd, uint Msg, WPARAM wParam, LPARAM lParam);

// This static method is required because Win32 does not support GetWindowLongPtr directly
private static IntPtr SetWindowLongPtr(HWND hWnd, WindowLongIndexFlags nIndex, IntPtr dwNewLong) =>
#if x64
    PInvoke.SetWindowLongPtr(hWnd, nIndex, dwNewLong);
#else
    PInvoke.SetWindowLong(hWnd, nIndex, dwNewLong.ToInt32());
#endif

private void SubClassingWin32()
{
    _hwnd = (HWND)WinRT.Interop.WindowNative.GetWindowHandle(this);

    WNDPROCDelegate del = NewWindowProc;
    var newWndProc = Marshal.GetFunctionPointerForDelegate(del);
    var oldWndProc = SetWindowLongPtr(_hwnd, WindowLongIndexFlags.GWL_WNDPROC, newWndProc);
    _oldWndProc = Marshal.GetDelegateForFunctionPointer<WNDPROC>(oldWndProc);
}

private LRESULT NewWindowProc(HWND hWnd, uint Msg, WPARAM wParam, LPARAM lParam)
{
    switch (Msg)
    {
        [...]
    }
    return PInvoke.CallWindowProc(_oldWndProc, hWnd, Msg, wParam, lParam);
}

The original code seems to use a trick were the 3rd parameter of SetWindowLongPtr is passes a delegate that is implicitly converted into a function pointer. With C#/Win32 this doesn't seem possible, so I'm using Marshal. The debugger shows NewWindowProc is being hit, so the first step seems to work, so I suspect something is going wrong in the line

_oldWndProc = Marshal.GetDelegateForFunctionPointer<WNDPROC>(oldWndProc);

or

PInvoke.CallWindowProc(_oldWndProc, hWnd, Msg, wParam, lParam);

But it's hard to debug (any tips there would also be welcome.)

Has anybody done this? Does anybody know what I'm doing wrong?

Answer 1

Well poop. The answer was looking me right into the eye.

I had already modified the original code, think I was smart and seeing the private field _newWndProc wasn't used anywhere. So I had made it a local variable.

and what happens then? You take a function pointer to a local delegate object, that is marked for finalization as soon as the scope ends... nothing is prolonging it's lifetime. I.e. the interop will use a pointer to an invalid object once it has been finilized!

So the solution is to undo my smart-ass optimization and make newWndProc a field again. (This has cost me some hours to debug!)