Private package was created and pip installed but cannot import with python

Question

I created a private package in TestPyPI

The package has successfully pip installed:

(base) my_user:Desktop$ python3 -m pip install --index-url https://test.pypi.org/simple/ --no-deps charter-common-utils==0.0.1
Looking in indexes: https://test.pypi.org/simple/
    Requirement already satisfied: charter-common-utils==0.0.1 in /Users/my_id/opt/anaconda3/lib/python3.7/site-packages (0.0.1)

I start python in terminal:

>>> import charter_common_utils
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'charter_common_utils'

I've read about the python path issues but that does not seem to be the issue since the last path listed is the one referred to in the 'Requirement already satisfied' above:

(base) SR-C02XT71WJG5J:Desktop p2929612$ python3
Python 3.7.6 (default, Jan  8 2020, 13:42:34) 
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/Users/my_id/opt/anaconda3/lib/python37.zip', '/Users/my_id/opt/anaconda3/lib/python3.7', '/Users/my_id/opt/anaconda3/lib/python3.7/lib-dynload', '/Users/my_id/.local/lib/python3.7/site-packages', '/Users/my_id/opt/anaconda3/lib/python3.7/site-packages']

When I follow /Users/my_id/opt/anaconda3/lib/python3.7/site-packages' I am able to see the file charter_common_utils-0.0.1.dist-info

Why am I not able to import the package? Any help is much appreciated.

Answer 1

Your setup.py lists a lot of top-level packages:

    packages=['anomaly', 'batch_transform', 'hive_table_checker', 'metadata_io',
              'parquet_converter', 'pyspark_visualizer'],

After installation you could import anomaly or parquet_converter but not charter_common_utils; the latter is nowhere mentioned. To import charter_common_utils you have to:

1) create a new directory charter_common_utils at the top of your source directory (where setup.py resides);

2) create a new empty file charter_common_utils/__init__.py;

3) move all your top-level directories (anomaly, batch_transform, hive_table_checker, metadata_io, parquet_converter, pyspark_visualizer) into charter_common_utils;

4) change your setup.py:

    packages=['charter_common_utils',
              'charter_common_utils.anomaly',
              'charter_common_utils.batch_transform',
              'charter_common_utils.hive_table_checker',
              'charter_common_utils.metadata_io',
              'charter_common_utils.parquet_converter',
              'charter_common_utils.pyspark_visualizer',
    ], 

Or change setup.py this way:

from setuptools import find_packages()

…

    packages=find_packages(),