I'm trying to debug an application compiled with Ninja.
I have my source code /usr/local/...project-src/
I have my build output located at /usr/local/...project-src/out/Debug/build
The compiled output includes debug information
file out/Debug/build
includes:
ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, with debug_info, not stripped
I'm able to add breakpoints when using relative paths:
cd /usr/local/...project-src
gdb
file out/Debug/build
b x/y.cc:34
# success
Breakpoint 1 at <mem-loc>: file ../../x/y.cc, line 34.
But when I use absolute paths, it fails
cd /usr/local/...project-src
gdb
file out/Debug/build
b /usr/local/...project-src/x/y.cc
# failure
No source file named /usr/local/...project/x/y.cc.
info source
prints No current source file.
dir
prints Source directories searched: /usr/local/...project-src/out/Debug:$cdir:$cwd
I've also tried:
b ../../x/y.cc
, I tried this since that's what the successful command outputs. Surprisingly, it didn't work which is really confusing me.cd
to various directories after starting gdbset substitute-path
and adding directories using dir
I'm hoping the solution is simple, since breakpoints, and variable values, and everything else works, just not with absolute paths.
Also worth noting, once I've successfully added a breakpoint to a file (using the relative path), the other paths also work (both the absolute path and the ../../x/y.cc
path).
Lastly, as to why I want absolute paths to work, I'm using CLion's remote-gdb configuration to connect to a gdbserver, and CLion is using absolute paths for whatever reason. Perhaps there is a way to configure CLion to just use the x/y.cc
instead? I'm running the gdbserver with gdbserver :2000 out/Debug/build
and configured CLion's target remote, symbol file, and sysroot. I've also tried setting the path mappings in CLion.
Edit, testing on a dummy HelloWorld project using g++ -g
instead of ninja
to build, I'm able to add breakpoints using absolute paths e.g. b /usr/local/...untitled/main.cpp:4
. So it seems to be, for some reason, gdb
supports full paths for the HelloWorld project built with g++, but not for the real project built with Ninja.
tldr, resolved using gdb --readnow
.
Per the comment suggestion I began digging into my build config. 2 things I noticed:
1) The issue disappeared if I built with less debug details. But then I wouldn't be able to inspect expressions & variables. So I thought the issue may somehow related to gdb not having enough memory or cache to load all the debug info. This sounds reasonable since the project source code contains 100,000's of files.
2) As I mentioned earlier, I could add breakpoints using absolute paths after I had added a breakpoint using a relative path to the same file.
I learnt of the info
, info set
, and info sources
commands. Although the outputs were the same between the light-weight debug build and the full debug built (step 1), I noticed that the output of info sources
changed in step 2. Before I had added the breakpoint using a relative path, `info sources would list all source files under 'Source files for which symbols will be read in on demand'. But after adding a relative breakpoint, a few of the files (I think the files on the current frame) would be added to the loaded source files.
So I went looking for a way to tell gdb to load all the source files and discovered the gdb --readnow
flag (or file <built-file> -readnow
with 1 dash -
) and though it prints a bunch of warning messages, it seems to resolve my issue.
That being said, I never discovered how to configure CLion to use the readnow
flag. The newest EAP (2019.3) release supposedly supports configuring .gdbinit
files individual per project, though I haven't tried this. I also don't know if readnow
can be configured in a .gdbinit
file since it's not a setting. I kind of circumvented the entire configuring CLion issue when I figured how to correctly configure custom build targets and applications in CLion during this investigation.
Edit,
Yet another workaround. If I cd
into /usr/local/...project-src/out/Debug
which contains the build file (as opposed to /usr/local/...project-src/
), then absolute paths work even without readnow
.