Search code examples
pythonctypesdylib

Does the order in which dynamic libraries are loaded matter?


I am on OS X and installed the Gtk+3 package with Homebrew.

brew install gtk+3

I can load the installed libraries in Python with the ctypes module.

$ python2.6
Python 2.6.9 (unknown, Oct 23 2015, 19:19:20)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import cdll
>>> cdll.LoadLibrary('/usr/local/lib/libatk-1.0.0.dylib')
<CDLL '/usr/local/lib/libatk-1.0.0.dylib', handle 7fbd10f1a250 at 10aa33210>
>>> cdll.LoadLibrary('/usr/local/lib/libglib-2.0.0.dylib')
<CDLL '/usr/local/lib/libglib-2.0.0.dylib', handle 7fbd10f0ffb0 at 10aa22dd0>
>>> ^D

So far so good. One thing that is bothering me is that if I try to load the same two libraries above, but in a different order, it throws a Symbol not found exception.

$ python2.6
Python 2.6.9 (unknown, Oct 23 2015, 19:19:20)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import cdll
>>> cdll.LoadLibrary('/usr/local/lib/libglib-2.0.0.dylib')
<CDLL '/usr/local/lib/libglib-2.0.0.dylib', handle 7fad13d00d60 at 10a688210>
>>> cdll.LoadLibrary('/usr/local/lib/libatk-1.0.0.dylib')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/ctypes/__init__.py", line 423, in LoadLibrary
    return self._dlltype(name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/ctypes/__init__.py", line 345, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(/usr/local/lib/libatk-1.0.0.dylib, 6): Symbol not found: _g_free
  Referenced from: /usr/local/lib/libatk-1.0.0.dylib
  Expected in: flat namespace
 in /usr/local/lib/libatk-1.0.0.dylib
>>> ^D

So, loading atk first, and then glib, works. The other way around does not. Can anyone explain this behaviour?


Solution

  • Here's the same problem shown at the command line level with C. Here, I've made a main.c which calls foo() in foo.so, which calls bar() in bar.so, which calls (for fun) g_free and g_malloc in glib.

    $ ls -Flas main.o libfoo.so libbar.so | cut -c3-13,34-37,51-
    -rwxrwxr-x 2976 libbar.so*    # has bar(), which calls g_free/g_malloc
    -rwxrwxr-x 3504 libfoo.so*    # has foo(), which calls bar()   
    -rw-rw-r-- 2864 main.o        # has main(), which calls foo()
    

    The correct linking command:

    $ gcc -o main main.o -L. -lfoo -lbar -lglib-2.0
    $ LD_LIBRARY_PATH=. ./main
    worked!
    

    Trying to flip the library loading order:

    $ gcc -o main main.o -L. -lbar -lfoo -lglib-2.0
    ./libfoo.so: undefined reference to `bar'
    collect2: error: ld returned 1 exit status
    

    Which is due to essentially the same issue you're seeing. The relevant section from "man ld" is this:

    If the archive defines a symbol which was undefined in some object which appeared before the archive on the command line, the linker will include the appropriate file(s) from the archive. However, an undefined symbol in an object appearing later on the command line will not cause the linker to search the archive again.

    So one needs to keep this in mind when loading dynamic libraries even through cdll, it seems.