How do I interact with Visual Studios native debugger when writing my own "mixed mode" debugger?

Question

I've spent the last few days searching google, blogs and MSDN looking for any small scrap of info on how "interop" or "mixed mode" debugging is implemented in Visual Studio.

I'm attempting to implement my own debugger for a custom VM (actually, it gets JIT-ed to native code, which means I'll have to rebuild callstacks for the normal native debugger whenever it enter's the JIT-ed code), but I can't find any info on how you can actually interact with VS's native debugger and do the fancy "native/managed" like transitions that VS's mixed mode debugging pulls off.

So far I've found a few things that provide useful tidbits, but not enough to actually interact with the debugger. The best, most useful articles I've found are:

• Mike Stall's various blog posts (mainly these two) while providing useful info and glimpses at the inner workings, they seem to be very heavily tied to .Net debugging.
• MSDN's Creating a Basic Debugger gives a nice overview of how MS implements it's debuggers and how you can implement one on the same tech stack. Unfortunately, this doesn't really provide any real info on how to pass data off to VS's native debugger. (Combined with Mike Stall's blogs, it seems like both debuggers would be waiting for the same events, so how do you actually splice the results together in a meaningful way?)
• The Debugger Engine API (aka DbgEng) documentation. I found this by running Dependency Walker on VS's native debug engine (NatDbgDE.dll, it seems to export the DebugCreate function which I could only find in this documentation, so it could be related). Unfortunately I've had very little success in even doing any debugging through this and it seems to be poorly documented, although the documentation does seem to indicate this is what I'll probably want to deal with in the end. It also doesn't really say how I can work cooperatively with another debugger and makes no mention of VS's debugger, so I could be going down the wrong path entirely with this anyway.

How can I get started with writing a debugger that will work cooperatively with VS's native debugger?

Thanks!

Answer 1

I'm afraid there's not much documentation on this subject. The resources you mention are all pretty old at this point. My suggestion is to integrate with Visual Studio's newer debug engine (Concord). Concord is used as the debug engine for native debugging from Visual Studio 2012 forward and for all debugging in Visual Studio 2013 forward. One of the design goals of Concord was to simplify mixed mode debugging. It is also designed to be easily extensible.

I created some Concord extensibility documentation that may be of help. It is mainly focused on expression evaluators, but it has some good information for getting started with Concord. Another resource that may be of use is the source code for the Concord-based debugger in Python Tools for Visual Studio. This is a full implementation of a mixed mode debugger that integrates with Concord and allows mixed mode debugging with Python.