Circular dependency issue with setup script

Question

I'm writing a python setup.py script for an own package which needs a version constant from the package. However, the package needs to have some dependencies installed. Therefore I specified install_requires in the setup.py. However, when I generate the package via python setup.py sdist and install it in another project I get a dependency error. What am I doing wrong, here? When I dismiss the import statements from the __init__.py file, the dependencies (in this case pyzmq and jsonpickle, but could be any other dependencies) are installed correctly in the other project.

folder-structure

myproject/
   | setup.py
   | mypackage/
     | __init__.py
     | some_code.py
     | version.py

__init__.py

from . import some_code
from . import version

some_code.py

import jsonpickle
import zmq

from mypackage.version import VERSION

print(f"Version is {VERSION}")

setup.py

import [...]
from distutils.dir_util import remove_tree
from setuptools import setup, find_packages

# Globals definitions used more than one time
PACKAGE_NAME = "mypackage"

[...]

 
from mypackage.version import VERSION  # <<<<---- This is the command which is problematic !!!!!


setup(name = PACKAGE_NAME,
    version = VERSION,
    author = "Me",
    author_email = "[email protected]",
    description='mypackage test',
    include_package_data = True,
    packages=find_packages(),
    install_requires=[
    'pyzmq', 'jsonpickle'
    ],
    zip_safe = False)

In another project

(.venv) user@l-user:~/PycharmProjects/using_mypackage$ python -m pip install mypackage-3.4.6.tar.gz 
Looking in indexes: https://pypi.org/simple
Processing ./mypackage-3.4.6.tar.gz
  Preparing metadata (setup.py) ... error
  ERROR: Command errored out with exit status 255:
   command: /home/user/PycharmProjects/using_mypackage/.venv/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-4upmamin/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-4upmamin/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-w7plcisg
       cwd: /tmp/pip-req-build-4upmamin/
  Complete output (9 lines):
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-req-build-4upmamin/setup.py", line 3, in <module>
      from mypackage.version import VERSION  # <<<<---- This is the command which is problematic !!!!!
    File "/tmp/pip-req-build-4upmamin/mypackage/__init__.py", line 1, in <module>
      from . import some_code
    File "/tmp/pip-req-build-4upmamin/mypackage/some_code.py", line 1, in <module>
      import jsonpickle
  ModuleNotFoundError: No module named 'jsonpickle'

Answer 1

The legacy "canonical" solution has been to use a regexp to find the line with VERSION and ast.literal_eval the string on that line with something like

with open(os.path.join(os.path.dirname(__file__), "mypackage", "__init__.py")) as infp:
    version = ast.literal_eval(
        re.search("^VERSION = (.+?)$", infp.read(), re.M).group(1)
    )

This works until it doesn't (but happily it will break loudly when someone tries to run setup.py).

However, if you don't need setup.py at all (though based on the fact that you're referring to remove_tree and probably using it in the part you've elided, you might?), you could switch altogether to having no imperative setup.py at all, by switching to PEP 517 builds; setuptools has nice machinery to let you just do version = attr:mypackage.VERSION.

packaging.python.org has a great step-by-step tutorial on that. You may also find my setuppy2cfg converter tool useful.