Keyboard hook changes the behavior of keys

Question

I'm creating a program that installs a keyboard hook to capture all keys and display some text related to them.

However, I've hit upon a snag, and that is some keys change behavior when the hook is installed.

I'll see about posting a small, but complete, test program, but for now I'll just describe the problem.

The problem exhibits itself on Windows 7 64-bit, in .NET 4.0, a C# program. I assume none of this matters.

My hook installs itself via SetWindowsHookEx and then handles all keys processed in the system.

If the hook method simply returns, or does minimal processing of the key (I'll post what changes the behavior in a second), the keyboard functions as expected in programs.

However, if I call this function, ToAscii from User32.dll, to figure out which key on my keyboard OemTilde or similar really is, then any key that "overlays the next key" stops functioning. I don't know the correct name for such keys, but the two apostrophe-types, `` and ´, as well as~and¨`, stops functioning.

For instance, if I hit ~ and then N, it displays as follows:

• Without keyboard hook installed: ñ
• With keyboard hook installed: n (notice no ~ above)

Does anyone know why this happens and how I can fix this problem?

For now I'll settle for just handling the keys correctly in other programs, even if that means that I won't be able to correctly detect the right key sequences in my own program.

Some more information:

If I call the ToAscii function as part of the hook method, then a different problem occurs. Keys like ¨ are processed twice, ie. if I hit the ¨ once, Notepad receives two ¨¨ characters, and hitting N now just adds the N.

However, if I use BeginInvoke to process the key on a separate thread, after returning from the keyboard hook method, the first problem occurs.

---

My program is probably a bit special in that:

• I don't use keyboard state (ie. the "Key state" 256-byte array I pass around is just full of 0's)
• I don't care about dead keys (in the sense that my program won't process them, I care just enough about them that I don't want my program to render them useless to the rest of the system)

As such, my code ended up looking as follows:

private bool IsDeadKey(uint key)
{
    return ((Hook.Interop.MapVirtualKey(key, 2) & 2147483648) == 2147483648);
}

void _Hook_KeyDown_Async(KeyDownEventArgs e)
{
    var inBuffer = new byte[2];
    char key = '\0';
    if (!IsDeadKey((uint)e.KeyCode))
    {
        int ascii = Hook.Interop.ToAscii((int) e.KeyCode,
                                            e.ScanCode,
                                            _KeyState,
                                            inBuffer,
                                            e.Flags);
        if (ascii == 1)
        {
            key = Char.ToUpper((char) inBuffer[0]);
        }
    }

    BeginInvoke(
        new Action<Keys, Boolean, Boolean, Boolean, Char>(ProcessKeyboardEvent),
        e.KeyCode, e.Control, e.Shift, e.Alt, key);
}

Answer 1

These keys are called dead keys and you might be able to solve the problem by removing the call to ToAscii. See also the following related thread:

ToAscii/ToUnicode in a keyboard hook destroys dead keys.

Update: I haven't seen your code, but when processing the parameters of the KeyboardProc callback function, can you check that you pass the keyboard message on when the code parameter is less than 0? The documentation says:

code [in] int

A code the hook procedure uses to determine how to process the message. If code is less than zero, the hook procedure must pass the message to the CallNextHookEx function without further processing and should return the value returned by CallNextHookEx.

There is a sample for setting up a managed hook in MSDN:

if (nCode < 0)
{
    return CallNextHookEx(hHook, nCode, wParam, lParam);
}
else
{
    // process message here

    return CallNextHookEx(hHook, nCode, wParam, lParam); 
}