Search code examples
pythonanacondasetuptoolspypi

Bad Python package organization installed a `tests` package to my path? (Anaconda)

Somehow (due to some bad advice) I believe have been bitten by using a shoddy organization of my project directory.

It currently looks like this (GOOD!):

\projectname
   setup.py
   LICENSE
   MANIFEST.in
   README.rst
   CHANGELOG.rst
   \src
      \projectname
         __init__.py
   \tests
      __init__.py
      test1.py

However, it used to look like this (BAD!): 

\package_name
   setup.py
   LICENSE
   MANIFEST.in
   README.rst
   CHANGELOG.rst
   \projectname <--- main package at top level
      __init__.py
   \tests
      test1.py

Prior to fixing my shoddy setup, I was using an editable install of the project (python setup.py develop). I have subsequently removed (pip uninstall my_package) that install (and re-added) several times.

Despite removing it, I have now discovered that I seem to have a package called tests available for importing! And the package path is pointed to the directory of my project: 

>>> import tests
>>> tests.__path__
_NamespacePath(['c:\\users\\username\\projects\\projectname\\tests'])

I am guessing this happened as a result of installing my package using the initial broken package organization.

How can I remove this tests "package" from my path? I am using Anaconda 4.3 (latest version). pip uninstall tests was not successful.

NOTE: My project has also been published to pypi (project name: parmatter) and I did install it once from there as a test, but only after using the modified project organization structure, so I don't think this was the cause. If anyone wants to install it and see if it leads to the same problem, that may at least help me nail down the cause.

Project repo: https://github.com/Ricyteach/parmatter

Source distribution: parmatter-0.0.5.tar.gz

EDIT

Result of python -m site below. 

C:\>python -m site
sys.path = [
    'C:\\',
    'C:\\Users\\ricky\\Anaconda3\\python36.zip',
    'C:\\Users\\ricky\\Anaconda3\\DLLs',
    'C:\\Users\\ricky\\Anaconda3\\lib',
    'C:\\Users\\ricky\\Anaconda3',
    'C:\\Users\\ricky\\Anaconda3\\lib\\site-packages',
    'C:\\Users\\ricky\\Anaconda3\\lib\\site-packages\\Sphinx-1.5.6-py3.6.egg',
    'c:\\users\\ricky\\projects\\parmatter',
    'C:\\Users\\ricky\\Anaconda3\\lib\\site-packages\\win32',
    'C:\\Users\\ricky\\Anaconda3\\lib\\site-packages\\win32\\lib',
    'C:\\Users\\ricky\\Anaconda3\\lib\\site-packages\\Pythonwin',
    'C:\\Users\\ricky\\Anaconda3\\lib\\site-packages\\setuptools-27.2.0-py3.6.egg',
]
USER_BASE: 'C:\\Users\\ricky\\AppData\\Roaming\\Python' (doesn't exist)
USER_SITE: 'C:\\Users\\ricky\\AppData\\Roaming\\Python\\Python36\\site-packages' (doesn't exist)
ENABLE_USER_SITE: True

It appears I need to remove the project directory from site... how do I do that? And how did it get there in the first place?

Solution

  • The issue was caused by OP installing an old version of the project in development mode, using the system Python interpreter instead of in a venv (hint: don't do that). That creates a line in easy-install.pth which makes project dir(s) available in sys.path.

    Locate the easy-install.pth file in your site-packages directory, and remove any lines starting like this:

    c:\\users\\username\\projects

    As for your project structure, I recommend you remove __init__.py from the tests subdirectory. It is not needed, and means you'll have to explicitly exclude tests if you're using find_packages() helper.

    I installed your v0.0.5 distribution, and the problem does not remain in that version. Your distribution is broken for Python 2, you should include python_requires='>=3' in the setup.py setup call so that appropriate metadata will be generated to prevent pip from attempting to install it on Python 2.