How can I get a pointer to a managed function to provide to unmanaged code as a callback?

Question

I am interested in displaying a really-old-style open file dialog (that is, one that does not act like Windows file explorer) in a C# application. I found this example for calling the legacy GetOpenFileName function via [DllImport] and extern. However, to actually get the non-Explorer behavior, the GOFN page indicates I also need to set a hook:

Windows continues to support the old-style Open dialog box for applications that want to maintain a user-interface consistent with the old-style user-interface. To display the old-style Open dialog box, enable an OFNHookProcOldStyle hook procedure and ensure that the OFN_EXPLORER flag is not set.

I modified the example and created the function below to be a OFNHookProcOldStyle callback, which, if I'm reading the documentation for such callbacks correctly, should not change any behavior by simply always returning 0 (process the message). Having such a hook set should disable the Explorer behavior in the absence of the OFN_EXPLORER flag to re-enable it.

public static UIntPtr SimpleOfnHookProcOldStyle(
    [In] IntPtr hdlg, [In] uint uimsg, [In] UIntPtr wParam, [In] IntPtr lParam)
{
    return UIntPtr.Zero;
}

I set the OFN_ENABLEHOOK flag like so:

int OFN_ENABLEHOOK = (int)new Int32Converter().ConvertFromString("0x00000020");
ofn.flags = OFN_ENABLEHOOK;

However, I also need to set the actual hook callback, which is an IntPtr. How can I get an IntPtr to my SimpleOfnHookProcOldStyle method so that I can set it as the hook?

Answer 1

You need to

1. Define a static delegate with the same signature and assign your static function to it.

public delegate UIntPtr Delegate_type(IntPtr hdlg, uint uimsg, UIntPtr wParam, IntPtr lParam);
public static Delegate_type native_callback = SimpleOfnHookProcOldStyle;

2. Use GCHandle.Alloc to prevent the garbage collector from deleting it.

GCHandle handle = GCHandle.Alloc(native_callback);

3. Use Marshal.GetFunctionPointerForDelegate to get an IntPtr to this static delegate.

IntPtr funcPtr = Marshal.GetFunctionPointerForDelegate(native_callback);

4. Call handle.Free() after the dialog is closed so you don't leak memory.