How can I make setup.py include a package in a subdirectory?

Question

I have a library git repo sub_lib that I cannot publish directly to a pypi repository for various reasons. Instead I'm using it as a git submodule of another library main_lib in the following structure:

my_repo/
  setup.py
  main_lib/
    __init__.py
  sub_lib/  # a git submodule
    setup.py
    sub_lib/
      __init__.py

main_lib needs to import sub_lib, so how can I configure my_repo's setup.py to include both main_lib and sub_lib as packages? In particular, is it possible to have setup.py include a package from a subdirectory (since it's in sub_lib/sub_lib/)?

Current setup.py:

from setuptools import setup, find_packages
setup(
  name='main-lib',
  ...,
  packages=find_packages(exclude=['tests*']),
  package_data={'main_lib': ['py.typed'], 'sub_lib': ['py.typed']},
)

Or is it better to work around this by using a symlink?

Answer 1

I was able to get this working with a symlink:

my_repo/
  setup.py
  main_lib/
    __init__.py
  sub_lib_repo/  # a git submodule
    setup.py
    sub_lib/
      __init__.py
  sub_lib -> ./sub_lib_repo/sub_lib

In my particular case, I needed sub_lib to also be a pip submodule of main_lib, which I was able get working with a 2nd symlink:

my_repo/
  setup.py
  main_lib/
    __init__.py
    sub_lib -> ../sub_lib_repo/sub_lib
  sub_lib_repo/  # a git submodule
    setup.py
    sub_lib/
      __init__.py
  sub_lib -> ./sub_lib_repo/sub_lib

In this way things like from main_lib.sub_lib.foo import bar work once main_lib is pip installed, and things like from sub_lib.foo import bar work (necessary imports within sub_lib). No changes to setup.py were required.