How does Windows (specifically, Vista) determine if my application is hung?

I have a problem very similar to the one described here: https://microsoft.public.win32.programmer.kernel.narkive.com/Ly2P8Yp2/prevent-vista-from-marking-my-application-as-non-responding

That thread suggests that Task Manager sends WM_NULL to the process and expects the process to consume this message within timeout limit (5 seconds?). When I google for "WM_NULL hung" there are many references to the same technique.

However I don't see any WM_NULL messages in the queue of my application while it works on a lengthy operation - I have a secondary thread that switches to the main thread every 0.5 sec and calls PeekMessage() looking for WM_NULL, and it doesn't find any!

So, what's the method that Windows (Vista) uses to determine if an application is hung?

Which messages should my application consume so that Windows thinks that the application is responsive?

MORE DETAILS :

Along with PeekMessage() looking for WM_NULL, we also call PeekMessage() for mouse events, since we also want to understand if the user picked certain area of the window, where a stop sign is drawn. If the area is picked, we set a flag that the lengthy operation in the main thread periodically checks, and will stop if the stop sign is picked. The problem with Vista is that when it declares application as unresponsive, it replaces its window with a ghost window - see description of PeekMessage() :

If a top-level window stops responding to messages for more than several seconds, the system considers the window to be not responding and replaces it with a ghost window that has the same z-order, location, size, and visual attributes. This allows the user to move it, resize it, or even close the application. However, these are the only actions available because the application is actually not responding. When an application is being debugged, the system does not generate a ghost window.

This ghost window doesn't allow mouse picks to go through to our window, because the window is not on screen anymore! So my goal is to prevent this ghost window from appearing in the first place...

AFTER SOME MORE INVESTIGATIONS :

After I added the code Michael suggested in his reply to this question

while (PeekMessage(&msg, NULL, 0, 0, PM_REMOVE))
{
    TranslateMessage(&msg);
    DispatchMessage(&msg);
}

the application is not considered hung by Windows anymore; however I cannot use this solution because application starts reacting to picks to various buttons, etc (which should not happen). So I tried to see which messages are coming in. I used Spy++ and also debug print, and both shown only two kinds of messages: WM_TIMER and 0x0118 (WM_SYSTIMER). So I modified the code like this

 while (PeekMessage(&msg, NULL, WM_TIMER, WM_TIMER, PM_REMOVE) ||
        PeekMessage(&msg, NULL, 0x0118, 0x0118, PM_REMOVE))
 {
    TranslateMessage(&msg);
    DispatchMessage(&msg);
 }

Surprisingly, the application hangs again!!

Now I'm really stuck. If I'm intercepting the only messages that come in, and let application process them, how come Windows still thinks that the application doesn't process events??

Any meaningful suggestion would be GREATLY appreciated.

Solution

TaskManager probably uses IsHungAppWindow to determine if an application is hung. Per MSDN, an application is considered hung if its not waiting for input, is not in startup processing, or has not not processed messages within 5 seconds. So no WM_NULL necessary.

You don't need to consume any specific messages - just regularly pump messages and move long tasks off of the UI thread. If you can get PeekMessage to be called every 0.5 seconds, replace it with something like:

while (PeekMessage(&msg, NULL, 0, 0, PM_REMOVE))
{
    TranslateMessage(&msg);
    DispatchMessage(&msg);
}

Which will completely drain your message queue and make you appear more responsive to the user. Do not filter individual messages such as mouse messages.. You should do this more than every 0.5 seconds if possible, and longer term try to move the long work off of the UI thread.