SetWindowsHookEx is always returning zero in C#

Question

I'm trying to hook a 3rd party app so that I can interact with the controls. But I am having issues with using SetWindowsHookEx to handle WH_KEYBOARD. It seems there is some problem with parameters that I am passing to SetWindowsHookEx.

public partial class Form1 : Form
{
    private delegate int HookProc(int code, IntPtr wParam, IntPtr lParam);
    static IntPtr hHook;
    IntPtr windowHandle;
    uint processHandle;

    HookProc PaintHookProcedure;

    [System.Runtime.InteropServices.DllImport("user32.dll", EntryPoint = "FindWindow", SetLastError = true)]
    static extern System.IntPtr FindWindowByCaption(int ZeroOnly, string lpWindowName);

    [System.Runtime.InteropServices.DllImport("user32.dll", EntryPoint = "SetWindowsHookEx", SetLastError = true)]
    static extern IntPtr SetWindowsHookEx(int idHook, HookProc lpfn, IntPtr hMod, uint dwThreadId);

    [System.Runtime.InteropServices.DllImport("user32.dll")]
    static extern int CallNextHookEx(IntPtr hhk, int nCode, IntPtr wParam, IntPtr lParam);

    // When you don't want the ProcessId, use this overload and pass IntPtr.Zero for the second parameter
    [System.Runtime.InteropServices.DllImport("user32.dll")]
    static extern uint GetWindowThreadProcessId(IntPtr hWnd, out uint lpdwProcessId);

    [System.Runtime.InteropServices.DllImport("kernel32.dll", CharSet = System.Runtime.InteropServices.CharSet.Auto)]
    public static extern IntPtr GetModuleHandle(string lpModuleName);

    public Form1()
    {
        InitializeComponent();
    }

    private void button1_Click(object sender, EventArgs e)
    {

        PaintHookProcedure = new HookProc(PaintHookProc);
        windowHandle = FindWindowByCaption(0, "Untitled - Notepad");
        uint threadID = GetWindowThreadProcessId(windowHandle, out processHandle);
        IntPtr hMod = System.Runtime.InteropServices.Marshal.GetHINSTANCE(typeof(Form1).Module);

        // HERE IS THE PROBLEM.  It returns always zero. No matter what parameters you pass.
        hHook = SetWindowsHookEx(WH_KEYBOARD, PaintHookProcedure, hMod, threadID);


    }

    public int PaintHookProc(int nCode, IntPtr wParam, IntPtr lParam)
    {
        // Do something here
        return CallNextHookEx(hHook, nCode, wParam, lParam);
    }

    private const int WH_KEYBOARD = 2;

}

Above is the sample code. SetWindowsHookEx is returning always zero. Any suggestions would be very helpful. Thanks in advance.

Answer 1

Did you look at this page: https://support.microsoft.com/en-us/help/318804/how-to-set-a-windows-hook-in-visual-c--net ?

Your Dllimports are subtly different

[System.Runtime.InteropServices.DllImport("user32.dll", EntryPoint = "SetWindowsHookEx", SetLastError = true)]
static extern IntPtr SetWindowsHookEx(int idHook, HookProc lpfn, IntPtr hMod, uint dwThreadId);

vs

[DllImport("user32.dll",CharSet=CharSet.Auto, CallingConvention=CallingConvention.StdCall)]
public static extern int SetWindowsHookEx(int idHook, HookProc lpfn, IntPtr hInstance, int threadId);