Recently I struggled porting an existing C++ application onto a new production environment (updated kernel, updated glibc, etc). Even though the output of ldd showed that all my .so were found, including the ELF interpreter, the execution always resulted on a "No such file or directory".
$ ldd my_app
libpthread.so.0 => /lib/libpthread.so.0 (0x00007fd41afd6000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fd41ae52000)
libm.so.6 => /lib/libm.so.6 (0x00007fd41ad11000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007fd41acf7000)
libc.so.6 => /lib/libc.so.6 (0x00007fd41ab3a000)
/lib64/ld-linux-x86-64.so.2 => /lib/ld-linux-x86-64.so.2 (0x00007fd4258b5000) <= ELF Interpreter
libconfig.so.9 => /usr/lib/libconfig.so.9 (0x00007fd41ab2c000)
libsensors.so.4 => /usr/lib/libsensors.so.4 (0x00007fd41ab1b000)
$ file my_app
ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=c8c7eeb2f6bdb96dab7b0cc9ad41aa6e3d610ec7, stripped
$ readelf -a my_app | grep "interpreter"
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
I didn't think my problem had anything to do with the interpreter because of this output of ldd. So I tried all other possible workarounds with no success. After much digging online I encountered a related problem that the solution suggested the creation of a symlink from the application's hard-coded path for the interpreter to the real path one. In my case:
ln -s /lib/ld-2.29.so /lib64/ld-linux-x86-64.so.2
This finally resolved the problem. At the end I was frustrated since I wasn't looking for the real problem and it took me too long to realize it because of the misleading output of ldd.
Why does ldd say that the interpreter is there even though it will not be loaded when you run the binary? Is my interpretation of the use/output of ldd mistaken?
onto a new production environment
Your new production environment installed ld-linux-x86-64.so.2 into /lib instead of /lib64. This is highly unusual and non-standard. Many binaries will fail to run on such a system. Whoever "made" this system, probably made a grave mistake.
Why does ldd say that the interpreter is there
The way ldd works is: it sets an environment variable LD_TRACE_OBJECTS=1 and invokes the real interpreter it was compiled with.
Since your system (erroneously) installed ld-linux into /lib, ldd dutifully invoked /lib/ld-linux-x86-64.so.2, which does exist, and actually told you so.
In general, ldd is not trustworthy, especially when several versions of GLIBC are installed on a single system.