I have a setup command defined like this for distutils (using py2app for Mac OS X, if it matters):
setup(...,
extensions=Extension('tracking_funcs',
['tracking_funcs/tracking_funcs.pyx'],
include_dirs=[numpyincludedirs,]),
Extension('_psutil_osx',
sources = ['psutil/_psutil_osx.c',
'psutil/_psutil_common.c',
'psutil/arch/osx/process_info.c'],
define_macros=[('PSUTIL_VERSION', int(get_psutilver().replace('.', '')))],
extra_link_args=['-framework', 'CoreFoundation',
'-framework', 'IOKit']),
Extension('_psutil_posix',
sources = ['psutil/_psutil_posix.c'])],
...)
It builds all three extensions correctly:
...
building 'tracking_funcs' extension
creating build/bdist.macosx-10.6-x86_64/temp.macosx-10.6-x86_64-2.7/tracking_funcs
/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c tracking_funcs/tracking_funcs.c -o build/bdist.macosx-10.6-x86_64/temp.macosx-10.6-x86_64-2.7/tracking_funcs/tracking_funcs.o
... (some compiler warnings) ...
42 warnings generated.
/usr/bin/clang -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -g build/bdist.macosx-10.6-x86_64/temp.macosx-10.6-x86_64-2.7/tracking_funcs/tracking_funcs.o -o build/bdist.macosx-10.6-x86_64/lib.macosx-10.6-x86_64-2.7/tracking_funcs.so
building '_psutil_osx' extension
creating build/bdist.macosx-10.6-x86_64/temp.macosx-10.6-x86_64-2.7/psutil
... (some compiler warnings) ...
2 warnings generated.
/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DPSUTIL_VERSION=400 -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c psutil/_psutil_common.c -o build/bdist.macosx-10.6-x86_64/temp.macosx-10.6-x86_64-2.7/psutil/_psutil_common.o
/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DPSUTIL_VERSION=400 -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c psutil/arch/osx/process_info.c -o build/bdist.macosx-10.6-x86_64/temp.macosx-10.6-x86_64-2.7/psutil/arch/osx/process_info.o
/usr/bin/clang -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -g build/bdist.macosx-10.6-x86_64/temp.macosx-10.6-x86_64-2.7/psutil/_psutil_osx.o build/bdist.macosx-10.6-x86_64/lib.macosx-10.6-x86_64-2.7/_psutil_osx.so -framework CoreFoundation -framework IOKit
building '_psutil_posix' extension
/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c psutil/_psutil_posix.c -o build/bdist.macosx-10.6-x86_64/temp.macosx-10.6-x86_64-2.7/psutil/_psutil_posix.o
/usr/bin/clang -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -g build/bdist.macosx-10.6-x86_64/temp.macosx-10.6-x86_64-2.7/psutil/_psutil_posix.o -o build/dist.macosx-10.6-x86_64/lib.macosx-10.6-x86_64-2.7/_psutil_posix.so
...
Then it copies the built binaries into the package destination, but it only copies two of the three extensions:
...
copying file /.../build/bdist.macosx-10.6-x86_64/lib.macosx-10.6-x86_64-2.7/tracking_funcs.so -> /.../dist/my.app/Contents/Resources/lib/python2.7/lib-dynload/tracking_funcs.so
copying file /.../build/bdist.macosx-10.6-x86_64/lib.macosx-10.6-x86_64-2.7/_psutil_posix.so -> /.../dist/my.app/Contents/Resources/lib/python2.7/lib-dynload/_psutil_posix.so
...
Then my application crashes because it can't find the third extension at runtime.
How can I debug this? Where does distutils get its dependency tree from, if not from the list of extensions I define? Perhaps the bug is in py2app rather than distutils itself?
Py2app uses modulegraph to recursively build a source tree containing all the dependencies of all the dependencies of your project. Running py2app with the debug-modulegraph flag set will print a bunch of debugging information to the console and drop into a breakpoint that allow you to browse the contents of the module graph:
$ python setup.py py2app --debug-modulegraph
...
(Pdb) for item in mf.flatten(): print item
This will probably show something like (MissingModule) psutil._psutil_osx in its output, which means modulegraph isn't able to find the import path for that extension.
Modulegraph exposes a public function called addPackagePath that will allow you to give it additional hints as to where it should look for files in particular packages. In this case, adding something like this in setup.py should resolve the issue:
from modulegraph import modulegraph
modulegraph.addPackagePath('psutil', 'build/bdist.macosx-10.6-x86_64/lib.macosx-10.6-x86_64-2.7/psutil/')