I'm trying to compile a program that depends on Caffe using Waf (not my program, just trying to build it). During linking, I was getting the error
/usr/bin/ld: cannot find -lcaffe
So I added the path to libcaffe.so to obj.libpath in wscript and it compiled. Yet when I run the compiled binary, I get an error telling me it can't find libcaffe.so. How is it possible that linking succeeds, yet the binary can't find the library?
How is it possible that linking succeeds, yet the binary can't find the library?
Linking with /usr/bin/ld is what's known as static linking (not to be confused with fully-static linking, achieved by using -static flag).
When your libraries are installed in a standard place on the system (e.g. in /usr/lib), everything just works. When you do not install libcaffe.so in a standard place, you need to know (more of) what you are doing.
You can direct the static linker to search additional directories for libraries, using the -L /path/to/dir linker flag. This allows you to achieve a successful static link, and gets you ready to the next step...
... which is called runtime loading. Runtime loading is performed by a completely different program (e.g. /lib/ld-linux.so.2 on Linux).
The runtime loader again would have no problem loading your program and libcaffe.so if the latter is installed in a standard location. But when it is not installed in a standard location, the loader needs to be told where to find libcaffe.so. There are several system-specific methods for doing so.
Many UNIX runtime loaders pay attention to LD_LIBRARY_PATH environment variable.
It may be possible to compile DT_RUNPATH or DT_RPATH into the program itself, with e.g. -Wl,-rpath=/path/to/dir or similar static linker argument.
On some systems, it is possible to modify the "system" locations (which are searched by default) by editing /etc/ld.so.conf or /etc/ld.so.conf.d/... files.