In my target ARM embedded system, all applications and the dependent libraries are available only as stripped versions. Most of the binaries are present in /bin, /usr/bin paths and libraries present in /lib, /usr/lib path are all stripped.
But with these I was not able to run Valgrind and GDB. As both needs debug symbol information which is present in non-stripped version. So, I copied all the non-stripped (bins and libs) from my dev server to target system in a different path say(/tmp/test)
Bins -- /tmp/test/usr/bin, /tmp/test/bin
Libs -- /tmp/test/usr/lib, /tmp/test/lib
When I tried to run the unstripped binary, the process snmp_agent got crashed.
$LD_LIBRARY_PATH=/tmp/test/usr/lib/:/tmp/test/lib/ /tmp/test/usr/bin/snmp_agent -p 8000 &
Even I tried running dynamic linker as below but this time linker crashed.
$/tmp/test/lib/ld-2.24.so /tmp/test/usr/bin/snmp_agent -p 8000 &
Coredump when doing export and executing non-stripped bin:
root@myway:~# export LD_LIBRARY_PATH=/tmp/test/usr/lib/:/tmp/test/lib/; /tmp/test/usr/bin/snmp_agent -p 8000 &
[1] 21218
root@myway:~#
[1]+ Segmentation fault (core dumped) /tmp/test/usr/bin/snmp_agent -p 8000
root@myway:~#
Getting coredump for default linux cmds after export:
root@myway:~# ls
Segmentation fault (core dumped)
root@myway:~# env
Segmentation fault (core dumped)
root@myway:~# pwd
/home/root
root@myway:~# date
Segmentation fault (core dumped)
Thanks @nidhoegger suggestion to use strace. Found that I missed symlink version of libs with strace output:
open("/tmp/test/usr/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/tmp/test/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/lib/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0\20=\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=427500, ...}) = 0
mmap2(NULL, 491640, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x76351000
mprotect(0x763b8000, 65536, PROT_NONE) = 0
mmap2(0x763c8000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x67000) = 0x763c8000
close(3) = 0
Symlink lib in regular system lib path(stripped):
root@myway:~# ls -l /lib/libm.so.6
lrwxrwxrwx 1 root root 12 Jun 22 11:30 /lib/libm.so.6 -> libm-2.24.so
I didnt copy the symlink version of unstripped libs to my path /tmp/test/usr/lib and /tmp/test/lib/
root@myway:~# ls -l /tmp/test/lib/libm.so.6
ls: /tmp/test/lib/libm.so.6: No such file or directory
root@myway:~# ls -l /tmp/test/lib/libm-2.24.so
-rwxrwx--- 1 root root 4885456 Jun 22 20:15 /tmp/test/lib/libm-2.24.so
Likewise my unstripped program is trying to load nearly 30 dependent libraries(verified this using ldd also) and all of which doesn't have symlink to original lib file. So the lack of symlink file is the reason behind the crash?
So the lack of symlink file is the reason behind the crash?
No, the reason behind the crash is that your unstripped binaries are coming from a different build.
If the difference was only due to stripping, you would be able to mix and match stripped and unstripped binaries freely, but you clearly can't.
The reason why all parts of GLIBC (the loader ld-linux, libc.so.6, libpthread.so.0, libdl.so.2, etc) must match exactly (must come from the same build) is explained here.