How to get proper backtrace in process signal handler (armv7-uclibc)?

I already did google many times to find right solution for backtrace() in signal handler and tried almost everything but I was not able to get the backtrace successfully in my signal handler - this is not SIGUSR1 handler.

enable UCLIBC_HAS_BACKTRACE=y in uclibc config and compiled it
verified that libubacktrace.so is created
compiled my application binaries with following options -g -rdynamic -fexception or -funwind-tables
The binary itself seems to be "stripped"

However, I was not able to get full backtrace from signal handler. Only function addresses which I've call in signal handler were printed.

If I use target-gdb binary and attach the process by using gdb --pid command, I was able to get the full backtrace properly.

Also, I tried pstack but (pstack-1.2 - tried arm-patch but it's horrible... nothing printed) not very helpful.

Any advice?

1) Compiler options in Makefile

CFLAGS += -g -fexceptions -funwind-tables -Werror $(WARN) ...

2) Code

The code is extremely simple.

#define CALLSTACK_SIZE 10

static void print_stack(void) {
    int i, nptrs;
    void *buf[CALLSTACK_SIZE + 1];
    char **strings;

    nptrs = backtrace(buf, CALLSTACK_SIZE);
    printf("%s: backtrace() returned %d addresses\n", __func__, nptrs);

    strings = backtrace_symbols(buf, nptrs);

    if(strings == NULL) {
        printf("%s: no backtrace captured\n", __func__);
        return;
    }

    for(i = 0; i < nptrs; i++) {
        printf("%s\n", strings[i]);
    }

    free(strings);
}

...
static void sigHandler(int signum)
{
    printf("%s: signal %d\n", __FUNCTION__, signum);
    switch(signum ) {
    case SIGUSR2:
        // told to quit
        print_stack();
        break;
    default:
        break;
    }
}

Solution

Read carefully signal(7) and signal-safety(7).

A signal handler is restricted to call (directly or indirectly) only async-signal-safe-functions (practically speaking, most syscalls(2) only) and backtrace(3) or even printf(3) or malloc(3) or free are not async-signal-safe. So your code is incorrect: the signal handler sigHandler is calling printf and indirectly (thru print_stack) free and they are not async-signal-safe.

So your only option is to use the gdb debugger.

Read more about POSIX signal.h & signal concepts. Practically speaking, the nearly only sensible thing a signal handler can do is set some global, thread-local, or static volatile sig_atomic_t flag, which has to be tested elsewhere. It could also directly write(2) a few bytes into a pipe(7), that your application would read elsewhere (e.g. in its event loop, if it is a GUI application).

You could also use Ian Taylor's libbacktrace from inside GCC (assuming your program is compiled with debug info, e.g. with -g). It is not guaranteed to work in signal handlers (since it is not using only async-signal-safe functions), but it is practically quite useful.

Notice that the kernel is setting a call frame (in the call stack) for sigreturn(2) when processing a signal.

You might also use (especially if your application is single-threaded) sigaltstack(2) to have an alternate signal stack. I'm not sure it would be helpful.

If you have an event loop, you might consider using the Linux specific signalfd(2) and ask your event loop to poll it. For SIGTERM or SIGQUIT or SIGALRM it is a quite useful trick.