Search code examples
pythonclinuxctypesdlopen

Loading C library in Python that dlopens another C library - unresolved shared symbol

I have a commercial c library (a.so) that has several functions. When you call the a.open() function, it performs a dlopen() call for another library dynamically. If calling a.open('b'), it will open b.so. If calling a.open('c'), it will open c.so.

The problem is that a.so and b.so share a global variable defined in a.so, but referenced by b.so (and c.so,etc.). I am able to load a.so correctly in python using ctypes and see all the symbols in Python. However, when I call a.open('b'), it attempts to load b.so but returns undefined symbol.

//a.c -source for a.so library
int aglobal = 0;
void open(char* lib)
{ dlopen(lib); }

//b.c - source for b.so library
extern int aglobal;

Here is my python code to load:

from ctypes import cdll
p = ctypes.CDLL('a.so')
p.open('b')

returns the error code: undefined symbol: aglobal

Some other notes:

  • files are linked with -fPIC -rdynamic -shared

  • When I write a C program that does the same as the python program, there is not a problem.

  • I've tried swig to wrap the library also, and numerous other things, build options, etc. but same results.

Is Python binding the symbols differently or something?

Solution

  • You need to use RTLD_GLOBAL when you load a.so.

    The object's symbols shall be made available for the relocation processing of any other object. In addition, symbol lookup using dlopen(0, mode) and an associated dlsym() allows objects loaded with this mode to be searched.

    The Linux man page is a bit more straightforward:

    The symbols defined by this shared object will be made available for symbol resolution of subsequently loaded shared objects.

    The python documentation describes the presence of the option but does not describe what it does.

    New in version 2.6: The use_last_error and use_errno optional parameters were added.

    ctypes.RTLD_GLOBAL
    Flag to use as mode parameter. On platforms where this flag is not available, it is defined as the integer zero.

    ctypes.RTLD_LOCAL
    Flag to use as mode parameter. On platforms where this is not available, it is the same as RTLD_GLOBAL.

    ctypes.DEFAULT_MODE
    The default mode which is used to load shared libraries. On OSX 10.3, this is RTLD_GLOBAL, otherwise it is the same as RTLD_LOCAL.

    From the documentation, it appears you are on a system where DEFAULT_MODE is the same as RTLD_LOCAL, which is the converse of RTLD_GLOBAL.