Search code examples
windowswindbgsymbolscrash-dumps

Why symbols are not getting resolved


I have got a dump file from a client, however, I am not able to resolve the symbols which is causing issue when making sense of stack traces. Below is what I have done. Can you please advise what I am doing wrong here?

0:12> !sym -noisy
noisy mode - symbol prompts on

0:12> .symfix
DBGHELP: Symbol Search Path: cache*;SRV*http://msdl.microsoft.com/download/symbols
DBGHELP: Symbol Search Path: cache*;SRV*http://msdl.microsoft.com/download/symbols
DBGHELP: Symbol Search Path: cache*;SRV*http://msdl.microsoft.com/download/symbols

0:12> .reload
..
DBGHELP: C:\ProgramData\dbg\sym\ntdll.pdb\743996EA223643DFA2C1AA9BFDDCFE502\ntdll.pdb already cached
DBGHELP: C:\ProgramData\dbg\sym\ntdll.pdb\743996EA223643DFA2C1AA9BFDDCFE502\ntdll.pdb already cached

DBGHELP: ntdll - public symbols  
        C:\ProgramData\dbg\sym\ntdll.pdb\743996EA223643DFA2C1AA9BFDDCFE502\ntdll.pdb
..............................................................
................................................................
................................................................
................................................................
.........................
Loading unloaded module list
.............
DBGHELP: C:\ProgramData\dbg\sym\mscorlib.pdb\03EA52B254BD4C79B589768D49ED6C2D1\mscorlib.pdb already cached
DBGHELP: C:\ProgramData\dbg\sym\mscorlib.pdb\03EA52B254BD4C79B589768D49ED6C2D1\mscorlib.pdb already cached
*** WARNING: Unable to verify checksum for mscorlib.ni.dll
DBGHELP: mscorlib_ni - public symbols  
        C:\ProgramData\dbg\sym\mscorlib.pdb\03EA52B254BD4C79B589768D49ED6C2D1\mscorlib.pdb


0:12> k
Child-SP          RetAddr           Call Site
00000000`05a6cd90 000007fe`2bd6b8c1 mscorlib_ni+0x468597
00000000`05a6ce20 00000001`91c67a70 0x000007fe`2bd6b8c1
00000000`05a6ce28 000007fe`890284c6 0x00000001`91c67a70
00000000`05a6ce30 000007fe`2bd6b8fd mscorlib_ni+0x4684c6
00000000`05a6cec0 00000001`91c78c98 0x000007fe`2bd6b8fd
00000000`05a6cec8 000007fe`890284c6 0x00000001`91c78c98
00000000`05a6ced0 000007fe`2bd6b8fd mscorlib_ni+0x4684c6
00000000`05a6cf60 00000001`91c78b90 0x000007fe`2bd6b8fd
00000000`05a6cf68 000007fe`890284c6 0x00000001`91c78b90
00000000`05a6cf70 000007fe`2bd6b8fd mscorlib_ni+0x4684c6
00000000`05a6d000 00000001`91c67a70 0x000007fe`2bd6b8fd
00000000`05a6d008 000007fe`890284c6 0x00000001`91c67a70
00000000`05a6d010 000007fe`2bd6b8fd mscorlib_ni+0x4684c6
00000000`05a6d0a0 00000001`91c67a70 0x000007fe`2bd6b8fd
00000000`05a6d0a8 000007fe`890284c6 0x00000001`91c67a70
00000000`05a6d0b0 000007fe`2bd6b8fd mscorlib_ni+0x4684c6
00000000`05a6d140 00000001`91c75348 0x000007fe`2bd6b8fd
00000000`05a6d148 000007fe`890284c6 0x00000001`91c75348
00000000`05a6d150 000007fe`2bd6b8fd mscorlib_ni+0x4684c6
00000000`05a6d1e0 00000001`91c67a70 0x000007fe`2bd6b8fd
00000000`05a6d1e8 000007fe`890284c6 0x00000001`91c67a70
00000000`05a6d1f0 000007fe`2bd6b8fd mscorlib_ni+0x4684c6
00000000`05a6d280 00000001`91c69eb0 0x000007fe`2bd6b8fd
00000000`05a6d288 000007fe`890284c6 0x00000001`91c69eb0
00000000`05a6d290 000007fe`2bd6b8fd mscorlib_ni+0x4684c6
00000000`05a6d320 00000001`91c67a70 0x000007fe`2bd6b8fd
00000000`05a6d328 000007fe`890284c6 0x00000001`91c67a70
00000000`05a6d330 000007fe`2bd6b8fd mscorlib_ni+0x4684c6
00000000`05a6d3c0 00000001`91c67a70 0x000007fe`2bd6b8fd
00000000`05a6d3c8 000007fe`890284c6 0x00000001`91c67a70
00000000`05a6d3d0 000007fe`2bd6b8fd mscorlib_ni+0x4684c6
00000000`05a6d460 00000001`91c67ba0 0x000007fe`2bd6b8fd
00000000`05a6d468 000007fe`890284c6 0x00000001`91c67ba0
00000000`05a6d470 000007fe`2bd6b8fd mscorlib_ni+0x4684c6
00000000`05a6d500 000007fe`2bc1aed8 0x000007fe`2bd6b8fd
00000000`05a6d508 000007fe`2b893680 0x000007fe`2bc1aed8
00000000`05a6d510 00000000`05a6d7b8 0x000007fe`2b893680
00000000`05a6d518 00000000`05a6d7b0 0x5a6d7b8
00000000`05a6d520 00000001`91af0110 0x5a6d7b0
00000000`05a6d528 00000001`00000000 0x00000001`91af0110
00000000`05a6d530 00000000`00000000 0x00000001`00000000

EDIT As per Hans suggestion in comments below, here is the managed stack trace

0:12> !CLRStack
        Child SP               IP Call Site
0000000005a6cd90 000007fe89028597 System.Collections.Generic.Dictionary`2[[System.__Canon, mscorlib],[System.__Canon, mscorlib]].Insert(System.__Canon, System.__Canon, Boolean)
0000000005a6ce20 000007fe2bd6b8c1 
DBGHELP: C:\ProgramData\dbg\sym\System.Data.Entity.pdb\8F8E35EC731B4EE9B4811CB80EE790B41\System.Data.Entity.pdb already cached
DBGHELP: C:\ProgramData\dbg\sym\System.Data.Entity.pdb\8F8E35EC731B4EE9B4811CB80EE790B41\System.Data.Entity.pdb already cached

DBGHELP: System_Data_Entity - public symbols  
        C:\ProgramData\dbg\sym\System.Data.Entity.pdb\8F8E35EC731B4EE9B4811CB80EE790B41\System.Data.Entity.pdb
System.Data.Query.InternalTrees.RuleProcessor.ApplyRulesToSubtree(System.Data.Query.InternalTrees.RuleProcessingContext, System.Collections.ObjectModel.ReadOnlyCollection`1<System.Collections.ObjectModel.ReadOnlyCollection`1<System.Data.Query.InternalTrees.Rule>>, System.Data.Query.InternalTrees.Node, System.Data.Query.InternalTrees.Node, Int32)
0000000005a6cec0 000007fe2bd6b8fd System.Data.Query.InternalTrees.RuleProcessor.ApplyRulesToSubtree(System.Data.Query.InternalTrees.RuleProcessingContext, System.Collections.ObjectModel.ReadOnlyCollection`1<System.Collections.ObjectModel.ReadOnlyCollection`1<System.Data.Query.InternalTrees.Rule>>, System.Data.Query.InternalTrees.Node, System.Data.Query.InternalTrees.Node, Int32)
0000000005a6cf60 000007fe2bd6b8fd System.Data.Query.InternalTrees.RuleProcessor.ApplyRulesToSubtree(System.Data.Query.InternalTrees.RuleProcessingContext, System.Collections.ObjectModel.ReadOnlyCollection`1<System.Collections.ObjectModel.ReadOnlyCollection`1<System.Data.Query.InternalTrees.Rule>>, System.Data.Query.InternalTrees.Node, System.Data.Query.InternalTrees.Node, Int32)
0000000005a6d000 000007fe2bd6b8fd System.Data.Query.InternalTrees.RuleProcessor.ApplyRulesToSubtree(System.Data.Query.InternalTrees.RuleProcessingContext, System.Collections.ObjectModel.ReadOnlyCollection`1<System.Collections.ObjectModel.ReadOnlyCollection`1<System.Data.Query.InternalTrees.Rule>>, System.Data.Query.InternalTrees.Node, System.Data.Query.InternalTrees.Node, Int32)
0000000005a6d0a0 000007fe2bd6b8fd System.Data.Query.InternalTrees.RuleProcessor.ApplyRulesToSubtree(System.Data.Query.InternalTrees.RuleProcessingContext, System.Collections.ObjectModel.ReadOnlyCollection`1<System.Collections.ObjectModel.ReadOnlyCollection`1<System.Data.Query.InternalTrees.Rule>>, System.Data.Query.InternalTrees.Node, System.Data.Query.InternalTrees.Node, Int32)
0000000005a6d140 000007fe2bd6b8fd System.Data.Query.InternalTrees.RuleProcessor.ApplyRulesToSubtree(System.Data.Query.InternalTrees.RuleProcessingContext, System.Collections.ObjectModel.ReadOnlyCollection`1<System.Collections.ObjectModel.ReadOnlyCollection`1<System.Data.Query.InternalTrees.Rule>>, System.Data.Query.InternalTrees.Node, System.Data.Query.InternalTrees.Node, Int32)
0000000005a6d1e0 000007fe2bd6b8fd System.Data.Query.InternalTrees.RuleProcessor.ApplyRulesToSubtree(System.Data.Query.InternalTrees.RuleProcessingContext, System.Collections.ObjectModel.ReadOnlyCollection`1<System.Collections.ObjectModel.ReadOnlyCollection`1<System.Data.Query.InternalTrees.Rule>>, System.Data.Query.InternalTrees.Node, System.Data.Query.InternalTrees.Node, Int32)
0000000005a6d280 000007fe2bd6b8fd System.Data.Query.InternalTrees.RuleProcessor.ApplyRulesToSubtree(System.Data.Query.InternalTrees.RuleProcessingContext, System.Collections.ObjectModel.ReadOnlyCollection`1<System.Collections.ObjectModel.ReadOnlyCollection`1<System.Data.Query.InternalTrees.Rule>>, System.Data.Query.InternalTrees.Node, System.Data.Query.InternalTrees.Node, Int32)
0000000005a6d320 000007fe2bd6b8fd System.Data.Query.InternalTrees.RuleProcessor.ApplyRulesToSubtree(System.Data.Query.InternalTrees.RuleProcessingContext, System.Collections.ObjectModel.ReadOnlyCollection`1<System.Collections.ObjectModel.ReadOnlyCollection`1<System.Data.Query.InternalTrees.Rule>>, System.Data.Query.InternalTrees.Node, System.Data.Query.InternalTrees.Node, Int32)
0000000005a6d3c0 000007fe2bd6b8fd System.Data.Query.InternalTrees.RuleProcessor.ApplyRulesToSubtree(System.Data.Query.InternalTrees.RuleProcessingContext, System.Collections.ObjectModel.ReadOnlyCollection`1<System.Collections.ObjectModel.ReadOnlyCollection`1<System.Data.Query.InternalTrees.Rule>>, System.Data.Query.InternalTrees.Node, System.Data.Query.InternalTrees.Node, Int32)
0000000005a6d460 000007fe2bd6b8fd System.Data.Query.InternalTrees.RuleProcessor.ApplyRulesToSubtree(System.Data.Query.InternalTrees.RuleProcessingContext, System.Collections.ObjectModel.ReadOnlyCollection`1<System.Collections.ObjectModel.ReadOnlyCollection`1<System.Data.Query.InternalTrees.Rule>>, System.Data.Query.InternalTrees.Node, System.Data.Query.InternalTrees.Node, Int32)
0000000005a6d500 000007fe2bd6b8fd System.Data.Query.InternalTrees.RuleProcessor.ApplyRulesToSubtree(System.Data.Query.InternalTrees.RuleProcessingContext, System.Collections.ObjectModel.ReadOnlyCollection`1<System.Collections.ObjectModel.ReadOnlyCollection`1<System.Data.Query.InternalTrees.Rule>>, System.Data.Query.InternalTrees.Node, System.Data.Query.InternalTrees.Node, Int32)
0000000005a6d5a0 000007fe2bd635c0 System.Data.Query.PlanCompiler.TransformationRules.Process(System.Data.Query.PlanCompiler.PlanCompiler, System.Data.Query.PlanCompiler.TransformationRulesGroup)
0000000005a6d610 000007fe2bb67e40 System.Data.Query.PlanCompiler.PlanCompiler.Compile(System.Collections.Generic.List`1<System.Data.Query.PlanCompiler.ProviderCommandInfo> ByRef, System.Data.Query.InternalTrees.ColumnMap ByRef, Int32 ByRef, System.Data.Common.Utils.Set`1<System.Data.Metadata.Edm.EntitySet> ByRef)
0000000005a6d6b0 000007fe2bb6723b System.Data.EntityClient.EntityCommandDefinition..ctor(System.Data.Common.DbProviderFactory, System.Data.Common.CommandTrees.DbCommandTree)
0000000005a6d850 000007fe2bb66ff8 System.Data.EntityClient.EntityProviderServices.CreateDbCommandDefinition(System.Data.Common.DbProviderManifest, System.Data.Common.CommandTrees.DbCommandTree)
0000000005a6d8a0 000007fe2bb66425 System.Data.Objects.Internal.ObjectQueryExecutionPlan.Prepare(System.Data.Objects.ObjectContext, System.Data.Common.CommandTrees.DbQueryCommandTree, System.Type, System.Data.Objects.MergeOption, System.Data.Objects.Span, System.Collections.ObjectModel.ReadOnlyCollection`1<System.Collections.Generic.KeyValuePair`2<System.Data.Objects.ObjectParameter,System.Data.Objects.ELinq.QueryParameterExpression>>, System.Data.Common.Utils.AliasGenerator)
0000000005a6d9b0 000007fe2b9df8cd System.Data.Objects.ELinq.ELinqQueryState.GetExecutionPlan(System.Nullable`1<System.Data.Objects.MergeOption>)
0000000005a6db00 000007fe3394ce5c System.Data.Objects.ObjectQuery`1[[System.Int32, mscorlib]].GetResults(System.Nullable`1<System.Data.Objects.MergeOption>)
0000000005a6db60 000007fe3394cd43 System.Data.Objects.ObjectQuery`1[[System.Int32, mscorlib]].System.Collections.Generic.IEnumerable<T>.GetEnumerator()
0000000005a6dbc0 000007fe41eb6645 
DBGHELP: C:\ProgramData\dbg\sym\System.Core.pdb\F0392B6A92EB48AABA178F73FAE390841\System.Core.pdb already cached
DBGHELP: C:\ProgramData\dbg\sym\System.Core.pdb\F0392B6A92EB48AABA178F73FAE390841\System.Core.pdb already cached

DBGHELP: System_Core - public symbols  
        C:\ProgramData\dbg\sym\System.Core.pdb\F0392B6A92EB48AABA178F73FAE390841\System.Core.pdb
System.Linq.Enumerable+<UnionIterator>d__88`1[[System.Int32, mscorlib]].MoveNext()
0000000005a6dc20 000007fe89113ed8 System.Collections.Generic.List`1[[System.Int32, mscorlib]]..ctor(System.Collections.Generic.IEnumerable`1<Int32>)
0000000005a6dc80 000007fe33837228 System.Linq.Enumerable.ToList[[System.Int32, mscorlib]](System.Collections.Generic.IEnumerable`1<Int32>)
0000000005a6dcc0 000007fe41eb127c 
SYMSRV:  C:\ProgramData\dbg\sym\SysData.Business.pdb\80279CEA7E444BCC81ADCA5E07EDD4831\SysData.Business.pdb not found
SYMSRV:  File: SysData.Business.pdb

SYMSRV:  Notifies the client application that a proxy has been detected.
SYMSRV:  Connecting to the Server: http://msdl.microsoft.com/download/symbols.
SYMSRV:  Successfully connected to the Server.
SYMSRV:  Sending the information request to the server.
SYMSRV:  Successfully sent the information request to the server.
SYMSRV:  Waiting for the server to respond to a request.
SYMSRV:  Successfully received a response from the server.
SYMSRV:  Closing the connection to the Server.
SYMSRV:  Successfully closed the connection to the Server.
SYMSRV:  Get File Path: /download/symbols/SysData.Business.pdb/80279CEA7E444BCC81ADCA5E07EDD4831/SysData.Business.pdb

SYMSRV:  Notifies the client application that a proxy has been detected.
SYMSRV:  Connecting to the Server: http://msdl.microsoft.com/download/symbols.
SYMSRV:  Successfully connected to the Server.
SYMSRV:  Sending the information request to the server.
SYMSRV:  Successfully sent the information request to the server.
SYMSRV:  Waiting for the server to respond to a request.
SYMSRV:  Successfully received a response from the server.
SYMSRV:  Closing the connection to the Server.
SYMSRV:  Successfully closed the connection to the Server.
SYMSRV:  Notifies the client application that a proxy has been detected.
SYMSRV:  Connecting to the Server: http://msdl.microsoft.com/download/symbols.
SYMSRV:  Successfully connected to the Server.
SYMSRV:  Sending the information request to the server.
SYMSRV:  Successfully sent the information request to the server.
SYMSRV:  Waiting for the server to respond to a request.

SYMSRV:  Successfully received a response from the server.
SYMSRV:  Closing the connection to the Server.
SYMSRV:  Successfully closed the connection to the Server.
SYMSRV:  Get File Path: /download/symbols/SysData.Business.pdb/80279CEA7E444BCC81ADCA5E07EDD4831/file.ptr

SYMSRV:  Notifies the client application that a proxy has been detected.
SYMSRV:  Connecting to the Server: http://msdl.microsoft.com/download/symbols.
SYMSRV:  Successfully connected to the Server.
SYMSRV:  Sending the information request to the server.
SYMSRV:  Successfully sent the information request to the server.
SYMSRV:  Waiting for the server to respond to a request.
SYMSRV:  Successfully received a response from the server.
SYMSRV:  Closing the connection to the Server.
SYMSRV:  Successfully closed the connection to the Server.
SYMSRV:  http://msdl.microsoft.com/download/symbols/SysData.Business.pdb/80279CEA7E444BCC81ADCA5E07EDD4831/SysData.Business.pdb not found
SYMSRV:  C:\ProgramData\dbg\sym\SysData.Business.pdb\80279CEA7E444BCC81ADCA5E07EDD4831\SysData.Business.pdb not found
*** ERROR: Module load completed but symbols could not be loaded for SysData.Business.dll
DBGHELP: SysData_Business - no symbols loaded
SysData.Business.DataCollector.PolicyExectuor.GetPolicySets(System.Collections.Generic.List`1<Int32>)
0000000005a6e050 000007fe3a17d05c SysData.Business.DataCollector.PolicyExectuor.GetLinks(System.Collections.Generic.List`1<Int32>)
0000000005a6e0f0 000007fe3a17aed3 SysData.Business.DataCollector.PolicyExectuor`2[[System.Int32, mscorlib],[System.Guid, mscorlib]].RunPolicy(System.Collections.Generic.List`1<Int32>)
0000000005a6e2c0 000007fe3a17a453 SysData.Business.DataCollector.PolicyExectuor.RunPolicy()
0000000005a6e350 000007fe3a179f7c SysData.Business.DataCollector.RunPolicies()
0000000005a6e3b0 000007fe3a179c1f SysData.Business.DataCollector.ProcessDatapoint()
0000000005a6e440 000007fe890639a5 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
0000000005a6e5a0 000007fe89063719 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
0000000005a6e5d0 000007fe8910be92 System.Threading.TimerQueueTimer.CallCallback()
0000000005a6e660 000007fe8910bc7e System.Threading.TimerQueueTimer.Fire()
0000000005a6e6d0 000007fe89092143 System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
0000000005a6e720 000007fe8909136a System.Threading.ThreadPoolWorkQueue.Dispatch()
0000000005a6ec88 000007fe8a25a7f3 [DebuggerU2MCatchHandlerFrame: 0000000005a6ec88] 
0000000005a6ee18 000007fe8a25a7f3 [ContextTransitionFrame: 0000000005a6ee18] 
0000000005a6f038 000007fe8a25a7f3 [DebuggerU2MCatchHandlerFrame: 0000000005a6f038]

Solution

  • k displays the native part of the call stack only. This does not resolve those parts of .NET which are JIT-compiled on the fly. To see the .NET callstack, use !clrstack of the SOS extension.

    To see both, .NET and native call stack, use the !mk (remember as "managed" k) from SOSEX. Another option is !dumpstack of SOS, but that's less readable.