Search code examples
windows-7vb6windbgcrash-dumpsprocdump

Windbg crash dump analysis


I'm having a hard time getting any meaningful information from a crash dump I created with ProcDump, but I'm pretty sure it's relevant to a seemingly random crash I've been having.

I have a VB6 application running on Windows 7 64-bit. Every once in a while, it crashes, leaving an entry in the error log that faults ntdll.dll but gives no more information than that. So, I have been running the process with SysInternals' ProcDump running to automatically create crash dumps for me.

I've been unable to re-create the crash in-house, so I was pretty sure that if I had a dump at all, it would tell me what the issue was. However, after running for most of a day, I see that ProcDump wrote several dumps already, although the program is still running fine. It does seem to point to issues with ntdll.dll, but I have no idea where to begin applying a fix for this.

Running !analyze -v on one of the dumps gives me the following:

*******************************************************************************
*                                                                             *
*                        Exception Analysis                                   *
*                                                                             *
*******************************************************************************


FAULTING_IP: 
+0
00000000 ??              ???

EXCEPTION_RECORD:  ffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 00000000
   ExceptionCode: 80000003 (Break instruction exception)
  ExceptionFlags: 00000000
NumberParameters: 0

FAULTING_THREAD:  000007c8

PROCESS_NAME:  application.exe

ERROR_CODE: (NTSTATUS) 0x80000003 - {EXCEPTION}  Breakpoint  A breakpoint has been reached.

EXCEPTION_CODE: (HRESULT) 0x80000003 (2147483651) - One or more arguments are invalid

NTGLOBALFLAG:  0

APPLICATION_VERIFIER_FLAGS:  0

APP:  application.exe

BUGCHECK_STR:  APPLICATION_FAULT_STATUS_BREAKPOINT_AFTER_CALL

PRIMARY_PROBLEM_CLASS:  STATUS_BREAKPOINT_AFTER_CALL

DEFAULT_BUCKET_ID:  STATUS_BREAKPOINT_AFTER_CALL

LAST_CONTROL_TRANSFER:  from 7754431f to 7752014d

STACK_TEXT:  
0382fdf4 7754431f 00000005 035e62c8 00000001 ntdll!ZwWaitForMultipleObjects+0x15
0382ff88 74cd339a 00000000 0382ffd4 77539ed2 ntdll!TppWaiterpThread+0x33d
0382ff94 77539ed2 035e6298 74e2a30c 00000000 kernel32!BaseThreadInitThunk+0xe
0382ffd4 77539ea5 775441f3 035e6298 00000000 ntdll!__RtlUserThreadStart+0x70
0382ffec 00000000 775441f3 035e6298 00000000 ntdll!_RtlUserThreadStart+0x1b


STACK_COMMAND:  ~0s; .ecxr ; kb

FOLLOWUP_IP: 
ntdll!ZwWaitForMultipleObjects+15
7752014d 83c404          add     esp,4

SYMBOL_STACK_INDEX:  0

SYMBOL_NAME:  ntdll!ZwWaitForMultipleObjects+15

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: ntdll

IMAGE_NAME:  ntdll.dll

DEBUG_FLR_IMAGE_TIMESTAMP:  4ce7ba58

FAILURE_BUCKET_ID:  STATUS_BREAKPOINT_AFTER_CALL_80000003_ntdll.dll!ZwWaitForMultipleObjects

BUCKET_ID:  APPLICATION_FAULT_STATUS_BREAKPOINT_AFTER_CALL_ntdll!ZwWaitForMultipleObjects+15

WATSON_STAGEONE_URL:  http://watson.microsoft.com/StageOne/BlackJack_exe/1_5_0_0/50227d4e/unknown/0_0_0_0/bbbbbbb4/80000003/00000000.htm?Retriage=1

Followup: MachineOwner

Can anyone point me in the right direction, in terms of making sense of this entry, and what I can do about it?


Solution

  • Just to make sure, I've performed some tests on my side, attaching to healthy processes and making dumps of just started process. In all cases output of !analyze -v was pretty similar to yours, except of the fact that my one is more verbose, I think it depends on debugger version.

    For example, here is the output that I've got after attaching to just started Paint:

    *******************************************************************************
    *                                                                             *
    *                        Exception Analysis                                   *
    *                                                                             *
    *******************************************************************************
    
    GetPageUrlData failed, server returned HTTP status 404
    URL requested:     http://watson.microsoft.com/StageOne/mspaint_exe/6_1_7600_16385/4a5bca29/ntdll_dll/6_1_7601_17725/4ec4aa8e/80000003/00050530.htm?Retriage=1
    
    FAULTING_IP: 
    ntdll!DbgBreakPoint+0
    00000000`76d90530 cc              int     3
    
    EXCEPTION_RECORD:  ffffffffffffffff -- (.exr 0xffffffffffffffff)
    ExceptionAddress: 0000000076d90530 (ntdll!DbgBreakPoint)
    ExceptionCode: 80000003 (Break instruction exception)
    ExceptionFlags: 00000000
    NumberParameters: 1
    Parameter[0]: 0000000000000000
    
    FAULTING_THREAD:  0000000000000cbc
    
    DEFAULT_BUCKET_ID:  STATUS_BREAKPOINT
    
    PROCESS_NAME:  mspaint.exe
    
    ERROR_CODE: (NTSTATUS) 0x80000003 - {EXCEPTION}  Breakpoint  A breakpoint has been reached.
    
    EXCEPTION_CODE: (HRESULT) 0x80000003 (2147483651) - One or more arguments are invalid
    
    EXCEPTION_PARAMETER1:  0000000000000000
    
    MOD_LIST: <ANALYSIS/>
    
    NTGLOBALFLAG:  0
    
    APPLICATION_VERIFIER_FLAGS:  0
    
    PRIMARY_PROBLEM_CLASS:  STATUS_BREAKPOINT
    
    BUGCHECK_STR:  APPLICATION_FAULT_STATUS_BREAKPOINT
    
    LAST_CONTROL_TRANSFER:  from 0000000076e37ef8 to 0000000076d90530
    
    STACK_TEXT:  
    
    
    FOLLOWUP_IP: 
    ntdll!DbgBreakPoint+0
    00000000`76d90530 cc              int     3
    
    SYMBOL_STACK_INDEX:  0
    
    SYMBOL_NAME:  ntdll!DbgBreakPoint+0
    
    FOLLOWUP_NAME:  MachineOwner
    
    MODULE_NAME: ntdll
    
    IMAGE_NAME:  ntdll.dll
    
    DEBUG_FLR_IMAGE_TIMESTAMP:  4ec4aa8e
    
    STACK_COMMAND:  ~8s ; kb
    
    FAILURE_BUCKET_ID:  STATUS_BREAKPOINT_80000003_ntdll.dll!DbgBreakPoint
    
    BUCKET_ID:  X64_APPLICATION_FAULT_STATUS_BREAKPOINT_ntdll!DbgBreakPoint+0
    
    WATSON_STAGEONE_URL:      http://watson.microsoft.com/StageOne/mspaint_exe/6_1_7600_16385/4a5bca29/ntdll_dll/6_1_7601_17725/4ec4aa8e/80000003/00050530.htm?Retriage=1
    
    Followup: MachineOwner
    ---------
    

    I've also taken a look at the explanation of ProcDump flags here: http://technet.microsoft.com/en-us/sysinternals/dd996900.aspx. It looks like using command line like

    C:\apps\procdump.exe -accepteula -e -h -n 10 -t -w application.exe
    

    you make procdump stop on every sign of hanging or exception without setting specific parameters, like memory usage number or procent of CPU usage.

    I would suggest to use DebugDiag, which provides nice UI, where you could configure rules describing when dump should be created. Here is an explanation from me, how to gather a dump when you have excessive memory usage problem, or high CPU usage:

    http://kate-butenko.blogspot.com/2012/06/how-to-gather-dump-with-debugdiag.html

    and here is another fine screenshot-based explanation, how to get a dump in DebugDiag for specific exception:

    http://blogs.msdn.com/b/kaushal/archive/2012/05/09/using-debugdiag-to-capture-a-dump-on-first-chance-exception.aspx

    From set of more lightweight tools you may also check the AdPlus tool (resides in C:\Program Files\Debugging Tools for Windows (x64) folder). I prefer DebugDiag, as it allows to catch specific types of exceptions.