How to intercept a window message in a shell extension

Question

I have a shell extension that needs to reload its configuration when a specific window message (custom message registered with RegisterWindowMessage) is broadcasted by another application.

I tried several approaches to intercept the message:

• Installing a window subclass callback on a window of Windows Explorer, using SetWindowSubclass. This works on Window 7, but not on Windows 8, because apparently DllMain is not called on the main thread, and SetWindowSubclass doesn't work from another thread. This is mentioned in the documentation:

  You cannot use the subclassing helper functions to subclass a window across threads

• Installing a hook for CALLWNDPROC, using SetWindowsHookEx. Because I don't want to slow down the whole system, I install the hook for a specific thread only (the explorer's main thread). This works on Windows 8, but not on Windows 7... I suspect this is because I'm hooking on the wrong thread, but I'm not sure. And anyway, this approach seems overly intrusive.

• Creating a message-only window to handle the message. This doesn't work at all, because message-only windows don't receive broadcasted messages.

Is there a reliable way to receive a window message in a shell extension?

A window message initially seemed to be the easiest way to notify the shell extension, but if you think another mechanism would be more appropriate, I'm open to suggestions.

Answer 1

Create a hidden window and listen for the message in its window procedure.

1. Register a window class that has default values for all the fields apart from the window procedure and class name. You don't need to specify anything else in the window class since the window will never be visible.
2. When you create the window, pass 0 for the window style. Specifically exclude WS_VISIBLE.
3. Pass 0 for the WndParent when you create the window. This will make it a top level window and so eligible to receive broadcast messages.