Search code examples
cperlldlibxml2dynamic-loading

LD_LIBRARY_PATH doesn't take effect for LibXML.so

XML::LibXML is a Perl library. LibXML.so is a part of the library. I'm trying to make XML::LibXML use custom built libxml2. But providing LD_LIBRARY_PATH doesn't make any difference:

$ ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
    libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f66af5e9000)

$ LD_LIBRARY_PATH=/home/yuri/_/libxml2/.libs ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
    libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f2d26ae3000)

$ ldd /usr/lib/python3.7/site-packages/libxml2mod.so | grep libxml2
    libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f878cbc8000)

$ LD_LIBRARY_PATH=/home/yuri/_/libxml2/.libs ldd /usr/lib/python3.7/site-packages/libxml2mod.so | grep libxml2
    libxml2.so.2 => /home/yuri/_/libxml2/.libs/libxml2.so.2 (0x00007f6f8f5d8000)

What am I doing wrong? Is there a way to handle it? The goal is to make Perl script use custom built libxml2 (to investigate some issue).

UPD

$ find /usr/lib -name libxml2.so.2
/usr/lib/libxml2.so.2
/usr/lib/vmware-installer/2.1.0/lib/lib/libxml2.so.2
/usr/lib/vmware-installer/2.1.0/lib/lib/libxml2.so.2/libxml2.so.2
/usr/lib/vmware-vmrc/5.5/lib/libxml2.so.2
/usr/lib/vmware-vmrc/5.5/lib/libxml2.so.2/libxml2.so.2

$ find /usr/local/lib -name libxml2.so.2
Solution

  • tl;dr

    ExtUtils::MakeMaker hardcodes paths of the shared libraries in the binaries by means of RPATH entry in .dynamic section. But you can preload the needed library,

    $ LD_PRELOAD=path/to/custom/libxml2/.libs/libxml2.so.2 ldd ./blib/arch/auto/XML/LibXML/LibXML.so

    or make the loader ignore RPATH entry,

    $ LD_LIBRARY_PATH=path/to/custom/libxml2/.libs /usr/lib/ld-linux-x86-64.so.2 --inhibit-rpath /abs/path/to/LibXML.so /abs/path/to/perl ./1.pl

    or convert RPATH entry into RUNPATH one,

    $ chrpath --convert blib/arch/auto/XML/LibXML/LibXML.so

    or delete RPATH entry,

    $ chrpath --delete blib/arch/auto/XML/LibXML/LibXML.so

    or specify XMLPREFIX variable,

    $ perl Makefile.PL XMLPREFIX=path/to/custom/libxml2/build

    Under the hood

    ExtUtils::Liblist::ext() takes a list of libraries to be linked with, like -lxml2 -lz -llzma -licui18n -licuuc -licudata -lm -ldl, and turns them it into four or five variables, that make their way into the generated Makefile. One of those is LD_RUN_PATH. It contains all the paths where the libraries where found.

    $is_dyna is true if the library is dynamic (not ends with .a). $is_perl is true if Perl was not built with it, and %ld_run_path_seen is to not duplicate values in LD_RUN_PATH.

    The part that calculates LD_RUN_PATH, that generates part of the Makefile with the variable, and the part that passes it to the linker.

    Consequences

    LD_RUN_PATH and in its turn RPATH entry in the binary make the loader search for libraries (at runtime) in the directories where they where found (at compile time).

    -rpath=dir Add a directory to the runtime library search path. This is used when linking an ELF executable with shared objects. All -rpath arguments are concatenated and passed to the runtime linker, which uses them to locate shared objects at runtime. The -rpath option is also used when locating shared objects which are needed by shared objects explicitly included in the link; see the description of the -rpath-link option. If -rpath is not used when linking an ELF executable, the contents of the environment variable "LD_RUN_PATH" will be used if it is defined.

    https://jlk.fjfi.cvut.cz/arch/manpages/man/core/binutils/ld.1.en

    If a shared object dependency does not contain a slash, then it is searched for in the following order:

    o Using the directories specified in the DT_RPATH dynamic section attribute of the binary if present and DT_RUNPATH attribute does not exist. Use of DT_RPATH is deprecated.

    o Using the environment variable LD_LIBRARY_PATH, unless the executable is being run in secure-execution mode (see below), in which case this variable is ignored.

    https://jlk.fjfi.cvut.cz/arch/manpages/man/core/man-pages/ld.so.8.en

    $ perl Makefile.PL
$ make

    Result,

    $ readelf --dynamic ./blib/arch/auto/XML/LibXML/LibXML.so | grep RPATH
 0x000000000000000f (RPATH)              Library rpath: [/usr/lib]

$ objdump -x ./blib/arch/auto/XML/LibXML/LibXML.so | egrep RPATH
  RPATH                /usr/lib

$ LD_LIBRARY_PATH=path/to/custom/libxml2/.libs ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
    libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f6cfabb2000)

    --dynamic - display contents of the .dynamic section, -x - display all headers.

    Solution

    One way to remedy this is to preload needed instance of libxml2,

    $ LD_PRELOAD=path/to/custom/libxml2/.libs/libxml2.so.2 ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
    path/to/custom/libxml2/build/lib/libxml2.so.2 (0x00007fe183aeb000)

    Another is to tell Makefile.PL search for libxml2 in the desired location (at compile time).

    $ perl Makefile.PL XMLPREFIX=path/to/custom/libxml2/build
$ make

    Result,

    $ readelf --dynamic ./blib/arch/auto/XML/LibXML/LibXML.so | grep RPATH
 0x000000000000000f (RPATH)              Library rpath: [path/to/custom/libxml2/build/lib:/usr/lib]

$ objdump -x ./blib/arch/auto/XML/LibXML/LibXML.so | egrep RPATH
  RPATH                path/to/custom/libxml2/build/lib:/usr/lib

$ ldd ./blib/arch/auto/XML/LibXML/LibXML.so | grep libxml2
    libxml2.so.2 => path/to/custom/libxml2/build/lib/libxml2.so.2 (0x00007fe183aeb000)

    On a side note

    Should you care to debug XML::LibXML, you can run make with OPTIMIZE variable (add -B to make make rebuild everything in case the library has already been built),

    $ make OPTIMIZE='-g3 -O0'

    There's also -O2, and no -g in LDDLFLAGS, but not sure if that matters much.

    Alternatively, WriteMakeFile takes OPTIMIZE parameter, so you can add a line here,

    'OPTIMIZE' => '-g3 -O0'

    Or you can add the variable here, and pass it to Makefile.PL,

    $ perl Makefile.PL OPTIMIZE='-g3 -O0'