setuptools.setup() install_requires not installing listed packages

Question

I'm using:

python 3.7.4
setuptools 50.3.0
pip 20.2.3

I have a public github repository that I'm trying to turn into a package to install. It needs requests to work properly.

The relevant files are organized like this:

package
    __init__.py
    __version__.py
    functions.py
setup.py

This is in each file:

# __init__.py

from .functions import first_function, second_function

# __version__.py

# version, title, etc. are defined here
__requests__ = 'requests>=2,<3'

# setup.py

import setuptools

from package.__version__ import __requests__, ...

setuptools.setup(
    ...,
    install_requires=[__requests__]
)

When I try to install with pip install git+https://github.com/<user>/<repository name>.git@<version tag>, I get a ModuleNotFoundError: No module named 'requests'.

I've already looked through several popular github repositories (like requests and pandas), the pypa sample project on github, the documentation for setuptools, and many github issues plus stackoverflow questions, but I still haven't been able to figure out what I'm doing wrong here.

Answer 1

importing your code during setup.py will fail because your dependencies won't be installed (essentially you've created a cycle from setup.py instal_requires -> your code -> requests needing to be installed)

there's ~2 approaches to solving this

one is to read your dependencies at runtime using importlib.metadata (this is what I'd suggest!)

if sys.version_info >= (3, 8):
    import importlib.metadata as importlib_metadata
else:
    import importlib_metadata  # pip install importlib-metadata backport

dist = importlib_metadata.distribution('yourpackage')
# this is a list for example
# >>> importlib.metadata.distribution('pre-commit').requires
# ['cfgv>=2.0.0', 'identify>=1.0.0', 'nodeenv>=0.11.1', 'pyyaml>=5.1', 'toml', 'virtualenv>=20.0.8', 'importlib-resources; python_version < "3.7"', 'importlib-metadata; python_version < "3.8"']
__requests__, = [req for req in dist.requires if req.startswith('requests>=')]

the other option is to read your __init__.py instead of importing it and parsing it manually:

for example:

import ast

with open('myproject/__init__.py', encoding='UTF-8') as f:
    for line in f:
        if line.startswith('__requests__ ='):
            requests_version = ast.literal_eval(line.split('=')[1])
            break
    else:
        raise AssertionError(f'__requests__ = not found in {f.name}')

I'd suggest the former personally though I've seen both approaches taken for various reasons