How does LLDB find libraries loaded at runtime?

I'm trying to debug a program that I attach LLDB to while it is running. After I attach it dlopen()'s a shared library. This shared library has full debugging information. However for some reason it does not show up in image list so I can't set breakpoints in it.

Annoyingly this used to work, so it is possible for it to work. But I changed how our build system works a while ago and apparently broke it. Git rebase would be very painful, so I'm trying to understand how it is going wrong.

My question is: how does LLDB learn about a dlopen() and add the target object to its image list, and are there any logs of this process to help me see why it is failing?

Solution

This process is platform dependent, since it relies on features of the native loader. On Darwin and Linux, the loader provides a "signaling function" that it calls when a load happens. The function does nothing, it's just a place where lldb can set a breakpoint to be notified about new shared library loads. You can see that breakpoint once you are running, for instance on Darwin the notification function is lldb_image_notifier (-i lists lldb "internal" breakpoints):

(lldb) break list -i
Current breakpoints:
Kind: shared-library-event
-3: name = 'lldb_image_notifier', module = dyld, locations = 1, resolved = 1, hit count = 1

  -3.1: where = dyld`lldb_image_notifier, address = 0x00000001934a30ac, resolved, hit count = 1

Once stopped, lldb either queries the debug stub (debugserver on Darwin) or introspects the loader state directly to find the new library loads.

The lldb logging channel for loader operations is lldb dyld so run:

(lldb) log enable lldb dyld

before running, and you'll see the traffic when lldb is informed of new shared library loads.

You might also find it helpful to turn on the gdb-remote packet log, which records the traffic between lldb and the debug stub. That's at a lower level than the dyld log and you have to study:

https://sourceware.org/gdb/current/onlinedocs/gdb.html/Remote-Protocol.html

to understand what it means, but it is still sometimes revealing.