My goal is to trap Ctrl+Shift+key sequence (where key will be defined by my app) when any running console process has keyboard focus. Then upon such key stroke I need to insert some specific text into that console app.
The question is: once injected into a target console process how do I intercept its keyboard keystrokes?
I was hoping that my test dll would be injected and called from within the process that was receiving the keyboard input at the time, but unfortunately it wasn't. It was always called from within my own process that called SetWindowsHookEx() to install the hook.
That caveat is clearly stated in the documentation:
This hook may be called in the context of the thread that installed it. The call is made by sending a message to the thread that installed the hook. Therefore, the thread that installed the hook must have a message loop.
This hook is called in the context of the thread that installed it. The call is made by sending a message to the thread that installed the hook. Therefore, the thread that installed the hook must have a message loop.
And since neither hook gives you process information, they will not help you know which process will receive the keystroke. The closest you can do inside these hooks is using GetForegroundWindow() and GetWindowThreadProcessId(), but that does not guarantee that the process ID you determine will be the same process ID that actually receives the keystroke.
Then my thinking was to install WH_CBT global hook instead and trap instances when nCode == HCBT_SETFOCUS for when any window will receive keyboard focus. Then I will somehow need to establish that that window belongs to a console app by its HWND. And then use Dll Insjection method to run my code in its process
Given an HWND, you can determine its thread ID and process ID using GetWindowThreadProcessId(), and determine if it is a console window by calling GetClassName() and see if it is "ConsoleWindowClass".
However, that only tells you whether the window itself is a console window. It does not tell you which process is currently displaying a UI inside that window. It might be:
cmd.exe itself (or whatever command processor is specified in the %COMSPEC% environment variable).
a console app that was started by cmd.exe, and thus interacts with the console window belonging to cmd.exe.
a console app that was started directly, and thus has a dedicated console window created by the OS for it.
a non-console process that created its own console window via AllocConsole().
a non-console process that is associated with another process's console window via AttachConsole().
I guess that doesn't really matter, though. If a console window belongs to cmd.exe, it is going to receive keystrokes first and then delegate them to whatever child process is currently displayed in the window. If it is a console dedicated to a another process, that process is going to receive keystrokes directly. So, either way, you should be able to hook the process ID that actually created the console HWND, since that should be the first process that will see the keystrokes.
The question I have though is once injected into a target console process how do I intercept its keyboard keystrokes?
You can still use a keyboard hook like before. You can create a keyboard hook that is associated with a specific thread ID, so use the thread ID that created the console window, as that is the only thread that can directly interact with the window and receive input messages for it. Just make sure your thread that is installing the keyboard hook has a message loop. Whether you inject that thread remotely into the console process, or leave it inside your own process, that is up to you. You should be able to receive the keyboard events either way.