Why LD_PRELOAD doesn't work for one of loaded shared libraries?

Question

I have an in-house shared library on RedHat Linux 5.0 that provides functions free and malloc:

>nm ./libmem_consumption.so | grep -P -e "\bfree\b|\bmalloc\b"
0000000000006540 T free
00000000000088a0 T malloc

This shared library is responsible for providing information about memory consumption of a process. Unfortunatelly there is a problem with this shared library when it is used with Apache httpd. When Apache httpd is run with this library I get a coredump in libc::free and a message that the pointer is invalid. The problem seems to be in http.so which is a shared library loaded by libphp5.so that is loaded by the httpd.

Actually when I do not load http.so everything is OK and there is no coredump. (Loading or not loading http.so is managed by the directive in a configuration file: extension=http.so) When I load http.so the httpd process coredumps.

httpd is lauched in this way:

LD_PRELOAD=./libmem_consumption.so ./bin/httpd -f config

and coredumps on exit.

When I set LD_BIND_NOW=1 and http.so is loaded I see (under gdb) that http.so has free@plt pointing to libc::free and in in other loaded libraries (for example libphp5.so) free@plt points to libmem_consumption.so::free. How that could be possible?

By the way when I export LD_DEBUG=all and save output to a file I see these lines for libphp5.so (which is also loaded):

 25788: symbol=free;  lookup in file=/apache2/bin/httpd [0]
 25788: symbol=free;  lookup in file=/apache2/ps/lib/libmem_consumption.so [0]
 25788: binding file /apache2/modules/libphp5.so [0] to /apache2/ps/lib/libmem_consumption.so [0]: normal symbol `free' [GLIBC_2.2.5]

And completely different for http.so:

 25825: symbol=free;  lookup in file=/apache2/ext/http.so [0]
 25825: symbol=free;  lookup in file=/apache2/ps/lib/libz.so.1 [0]
 25825: symbol=free;  lookup in file=/apache2/ps/lib/libcurl.so.4 [0]
 25825: symbol=free;  lookup in file=/lib64/libc.so.6 [0]
 25825: binding file /apache2/ext/http.so [0] to /lib64/libc.so.6 [0]: normal symbol `free'

It seems that LD_PRELOAD=./libmem_consumption.so is not used for http.so when free is looked up. Why LD_PRELOAD is ignored?

Answer 1

It seeems that http.so is loaded with the RTLD_DEEPBIND flag and that is why LD_PRELOAD is ignored for one of shared libraries.

This is from http://linux.die.net/man/3/dlopen:

RTLD_DEEPBIND (since glibc 2.3.4) Place the lookup scope of the symbols in this library ahead of the global scope. This means that a self-contained library will use its own symbols in preference to global symbols with the same name contained in libraries that have already been loaded. This flag is not specified in POSIX.1-2001.

I wrote a test shared library:

  #include <dlfcn.h>
  #include <unistd.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>

  static void initialize_my_dlopen(void) __attribute__((constructor));

  void* (*real_dlopen)(const char *, int flag);
  static int unset_RTLD_DEEPBIND=0;
  static int _initialized = 0;

  static void initialize_my_dlopen(void)
  {
    if (_initialized)
        return;
    real_dlopen  = (void *(*)(const char *,int))dlsym(RTLD_NEXT, "dlopen");
    unset_RTLD_DEEPBIND = atoi(getenv("UNSET_RTLD_DEEPBIND") ? getenv("UNSET_RTLD_DEEPBIND") : "0");
    printf("unset_RTLD_DEEPBIND: %d\n", unset_RTLD_DEEPBIND);
    _initialized = 1;
  }

  extern "C" {

    void *dlopen(const char *filename, int flag)
    {
      int new_flag = unset_RTLD_DEEPBIND == 0 ? flag : flag & (~RTLD_DEEPBIND);
      return (*real_dlopen)(filename, new_flag);
    }
  }

And built it:

  gcc -shared -fPIC -g -m64 my_dlopen.cpp -o libmy_dlopen.so -ldl

When I set UNSET_RTLD_DEEPBIND to 0 and run httpd the program coredumps.

export UNSET_RTLD_DEEPBIND=0
LD_PRELOAD=./libmy_dlopen.so:./ps/lib/libmem_consumption.so ./bin/httpd -f config  

When I set UNSET_RTLD_DEEPBIND to 1 and run httpd everything is OK.

export UNSET_RTLD_DEEPBIND=1
LD_PRELOAD=./libmy_dlopen.so:./ps/lib/libmem_consumption.so ./bin/httpd -f config  

And this is the output of LD_DEBUG=all for the UNSET_RTLD_DEEPBIND to 1:

 10678: symbol=free;  lookup in file=/apache2/bin/httpd [0]
 10678: symbol=free;  lookup in file=/apache2/libmy_dlopen.so [0]
 10678: symbol=free;  lookup in file=/apache2/ps/lib/libmem_consumption.so [0]
 10678: binding file /apache2/ext/http.so [0] to /apache2/ps/lib/libmem_consumption.so [0]: normal symbol `free'