Run a shell script during pip install

Question

I have published a PyPI package which requires some additional Apt packages and so I have a setup.sh in my Github repository which I want to execute along with the installation process of pip install mypackage (i.e, whenever a user runs pip install mypackage it should run the shell script during the process of installation.)

I made my setup.py to use the cmdclass parameter like this

from setuptools import setup, find_packages
from setuptools.command.install import install
import subprocess
import codecs 
import os

class CustomInstall(install):
    def run(self):
        subprocess.check_call("bash setup.sh", shell=True)
        install.run(self)

with open('requirements.txt') as f:
    requirements = f.read().splitlines()


setup(
    ...
    cmdclass={'install': CustomInstall},   install_requires=requirements,
    ...
)

When I normally build it with python setup.py sdist bdist_wheel the Custominstall class runs fine but during pip installation it ignores the Custominstall class.

Finally I found this stackoverflow thread from where I learnt that I only need to run python setup.py sdist which I did and got a tar.gz file in the dist folder but when I go to install that tar.gz file with pip install, I am getting

with open('requirements.txt') as f:

           ^^^^^^^^^^^^^^^^^^^^^^^^

  FileNotFoundError: [Errno 2] No such file or directory: 'requirements.txt'

What am I doing wrong? Is the path not correct? The requirements file is in my repository's parent folder (not inside any other folder)

Answer 1

The reason why this is happening is because your setup.py depends on requirements.txt but you're not packaging requirements.txt in your distribution. If you open your .tar.gz, you won't find your requirements.txt file. To fix this, create a MANIFEST.in file with the contents:

include requirements.txt

And add include_package_data=True to your setup call.

You'll need to add any other non-python files you want to include in your .tar.gz distribution, such as your setup.sh script as well.

When you build your source distribution with these changes, you'll see that requirements.txt is now included in the contents of your build distribution.

Alternatively, you can also use the package_data keyword in setup for this (or [options.package_data] in setup.cfg). See: this link for more in depth explanation.

---

Generally speaking though, it's not a good idea to run shell scripts or install system packages as part of your setup.py. It will make your package harder to use, not easier.

If your package depends on Python dependencies use install_requires instead.