How to make setuptools follow symlinks?

Question

I'm trying to set up a python package for distribution. I haven't done this before, so I'm pretty hazy about best practices, or even how to make things work, and I've run into a problem. My project structure looks like this:

ROIWidgets/
    MANIFEST.in
    setup.py
    setup.cfg
    ROIWidgets/
        __init__.py
    static/
        widgets/
            js -> <symlink to folder containing JavaScript files>

The problem is with the JS files under static. I started with distutils. I included this line in MANIFEST.in:

recursive-include static *

...and that caused the JS to be included in the package. Nice. Then I read that if I want my setup script to automatically install dependencies, I need to use setuptools rather than distutils. So, I switched: I import setup from setuptools rather than distutils. But now, when I say setup.py sdist, this line appears in the output:

warning: no files found matching '*' under directory 'static'

...and the JS files are not included in the distribution. After a long session with the debugger, I located the problem in setuptools/__init__.py. Here it is:

def findall(dir = os.curdir):
    """Find all files under 'dir' and return the list of full filenames
    (relative to 'dir').
    """
    all_files = []
    for base, dirs, files in os.walk(dir):
        if base==os.curdir or base.startswith(os.curdir+os.sep):
            base = base[2:]
        if base:
            files = [os.path.join(base, f) for f in files]
        all_files.extend(filter(os.path.isfile, files))
    return all_files

distutils.filelist.findall = findall    # fix findall bug in distutils.

os.walk does not by default follow symlinks. So the setuptools version of findall isn't looking in my js directory. distutils findall didn't use os.walk -- instead it had its own directory walking code. From the changelog, it appears that distutils findall had a problem with broken symlinks, which this replacement was intended to address. It would have been easy enough to use os.walk(dir, followlinks=True), but perhaps there was some reason why that was a bad idea?

So I'm kind of stuck. Either I change the structure of my project, which I'm loathe to do, or I muck with distutils and setuptools internals, a prospect that makes me even less happy.

Any suggestions?

Answer 1

This was apparently fixed in the repository at the time of your question (possibly too early for the fix to have become widespread).

I believe this means the relevant version tag for a fixed setuptools package is 8.0.2. Updating to that version solves the issue.