Search code examples
c++debugginggdbreverse-engineeringdisassembly

Meaning of a gdb backtrace when there is not source code


I have a gdb backtrace of a crashed process, but I can't see the specific line in which the crash occurred because the source code was not in that moment. I don't understand some of the information given by the mentioned backtrace.

The backtrace is made of lines like the following one:

<path_to_binary_file>(_Z12someFunction+0x18)[0x804a378]

Notice that _Z12someFunction is the mangled name of int someFunction(double ).

My questions are:

Does the +0x18 indicate the offset, starting at _Z12someFunction address, of the assembly instruction that produced the crash?

If the previous question is affirmative, and taking into account that I am working with a 32-bit architecture, does the +0x18 indicates 0x18 * 4 bytes?

If the above is affirmative, I assume that the address 0x804a378 is the _Z12someFunction plus 0x18, am I right?

EDIT:

The error has ocurred in a production machine (no cores enabled), and it seems to be a timing-dependant bug, so it is not easy to reproduce it. That is because the information I am asking for is important to me in this occasion.


Solution

  • Most of your assumptions are correct. The +0x18 indeed means offset (in bytes, regardless of architecture) into the executable.

    0x804a378 is the actual address in which the error occurred.

    With that said, it is important to understand what you can do about it.

    First of all, compiling with -g will produce debug symbols. You, rightfully, strip those for your production build, but all is not lost. If you take your original executable (i.e. - before you striped it), you can run: addr2line -e executable

    You can then feed into stdin the addresses gdb is giving you (0x804a378), and addr2line will give you the precise file and line to which this address refers.

    If you have a core file, you can also load this core file with the unstriped executable, and get full debug info. It would still be somewhat mangled, as you're probably building with optimizations, but some variables should, still, be accessible.

    Building with debug symbols and stripping before shipping is the best option. Even if you did not, however, if you build the same sources again with the same build tools on the same environment and using the same build options, you should get the same binary with the same symbols locations. If the bug is really difficult to reproduce, it might be worthwhile to try.

    EDITED to add

    Two more important tools are c++filt. You feed it a mangled symbol, and produces the C++ path to the actual source symbol. It works as a filter, so you can just copy the backtrace and paste it into c++filt, and it will give you the same backtrace, only more readable.

    The second tool is gdb remote debugging. This allows you to run gdb on a machine that has the executable with debug symbols, but run the actual code on the production machine. This allows live debugging in production (including attaching to already running processes).