python distutils not include the SWIG generated module

Question

I am using distutils to create an rpm from my project. I have this directory tree:

project/
        my_module/
                 data/file.dat
                 my_module1.py
                 my_module2.py
        src/
            header1.h
            header2.h
            ext_module1.cpp
            ext_module2.cpp
            swig_module.i
        setup.py
        MANIFEST.in
        MANIFEST

my setup.py:

from distutils.core import setup, Extension

module1 = Extension('my_module._module',
                sources=['src/ext_module1.cpp',
                         'src/ext_module2.cpp',
                         'src/swig_module.i'],
                swig_opts=['-c++', '-py3'],
                include_dirs=[...],
                runtime_library_dirs=[...],
                libraries=[...],
                extra_compile_args=['-Wno-write-strings'])

setup(  name            = 'my_module',
        version         = '0.6',
        author          = 'microo8',
        author_email    = 'magyarvladimir@gmail.com',
        description     = '',
        license         = 'GPLv3',
        url             = '',
        platforms       = ['x86_64'],
        ext_modules     = [module1],
        packages        = ['my_module'],
        package_dir     = {'my_module': 'my_module'},
        package_data    = {'my_module': ['data/*.dat']} )

my MANIFEST.in file:

include src/header1.h
include src/header2.h

the MANIFEST file is automatically generated by python3 setup.py sdist. And when i run python3 setup.py bdist_rpm it compiles and creates correct rpm packages. But the problem is that when im running SWIG on a C++ source, it creates a module.py file that wraps the binary _module.cpython32-mu.so file, it is created with the module_wrap.cpp file, and it isnt copied to the my_module directory.

What I must write to the setup.py file to automatically copy the SWIG generated python modules?

And also I have another question: When I install the rpm package, I want that an executable will be created, in /usr/bin or so, to run the application (for example if the my_module/my_module1.py is the start script of the application then I can run in bash: $ my_module1).

Answer 1

The problem is that build_py (which copies python sources to the build directory) comes before build_ext, which runs SWIG.

You can easily subclass the build command and swap around the order, so build_ext produces module1.py before build_py tries to copy it.

from distutils.command.build import build

class CustomBuild(build):
    sub_commands = [
        ('build_ext', build.has_ext_modules), 
        ('build_py', build.has_pure_modules),
        ('build_clib', build.has_c_libraries), 
        ('build_scripts', build.has_scripts),
    ]

module1 = Extension('_module1', etc...)

setup(
    cmdclass={'build': CustomBuild},
    py_modules=['module1'],
    ext_modules=[module1]
)

However, there is one problem with this: If you are using setuptools, rather than just plain distutils, running python setup.py install won't run the custom build command. This is because the setuptools install command doesn't actually run the build command first, it runs egg_info, then install_lib, which runs build_py then build_ext directly.

So possibly a better solution is to subclass both the build and install command, and ensure build_ext gets run at the start of both.

from distutils.command.build import build
from setuptools.command.install import install

class CustomBuild(build):
    def run(self):
        self.run_command('build_ext')
        build.run(self)


class CustomInstall(install):
    def run(self):
        self.run_command('build_ext')
        self.do_egg_install()

setup(
    cmdclass={'build': CustomBuild, 'install': CustomInstall},
    py_modules=['module1'],
    ext_modules=[module1]
)

It doesn't look like you need to worry about build_ext getting run twice.