Search code examples
pythonsetuptoolspypidevpi

How to disable uploading a package to PyPi unless --public is passed to the upload command

I'm developing packages and uploading development/testing/etc versions of my packages to a local devpi server.

In order to prevent an accidental upload to PyPi, I'm adopted the common practice of:

setup(...,
      classifiers=[
        "Programming Language :: Python",
        "Programming Language :: Python :: 2",
        "Programming Language :: Python :: 2.7",
        "Private :: Do not Upload"
     ],
     ...)

which works great, but what about when I'm finally ready to upload the package to PyPi?

I've come up with a totally ugly, but simple hack which requires that I define the classifiers as a global variable outside of the setup() call which looks like:

CLASSIFIERS = [
    "Programming Language :: Python",
    "Programming Language :: Python :: 2",
    "Programming Language :: Python :: 2.7"
]


if "--public" not in sys.argv:
     CLASSIFIERS.append("Private :: Do Not Upload")
else:
     sys.argv.remove("--public")

setup(...
      classifiers=CLASSIFIERS,
      ...)

Another, and perhaps simpler option is to merely comment out the "Private :: Do not Upload", but that doesn't seem any more professional than my hack.

What I'd like to do is create a proper subclass of the upload command called SafeUpload and have it check for the --public cmd-line option. Perhaps, as a build may exist prior to uploading, SafeBuild might be a better option.

Unfortunately, I'm having trouble understanding the setuptools documentation on creating custom commands.

Does anyone have any idea how to implement this? It's not clear to me if a custom command has access to the parameters passed to setup(), i.e. could it directly manipulate the classifiers passed to setup(), or would if it require that a user of the command follow the convention of defining of CLASSIFIERS as a global variable yuck?

Solution

  • Going backwards on your questions; while it's really broad, the topic is still constrained enough.

    I can tell you that the classifiers are not manipulated, but rather read from the and then written to PKG-INFO file by the egg_info command, which in turn looks for all egg_info.writers entry_points which the setuptools.command.egg_info:write_pkg_info function will do the actual writing. As far as I can tell, trying to leverage that Classifier outside will not be a great way, however you can override everything and anything you want through setuptools so you can make your own write_pkg_info function, figure out how to read the metadata (which you can see in the main distutils.command.upload:upload.upload_file method) and manipulate that further before upload_file finally reads it. At this point you probably are thinking that manipulating and working with this system is going to be rather annoying.

    As I mentioned though, everything can be overridden. You can make an upload command that take the public flag, like so:

    from distutils.log import warn
from distutils.command.upload import upload as orig
# alternatively, for later versions of setuptools:
# from setuptools.command.upload import upload as orig

class upload(orig):
    description = "customized upload command"

    user_options = orig.user_options + [
        ('public', None,
         'make package public on pypi'),
    ]

    def initialize_options(self):
        orig.initialize_options(self)
        self.public = False

    def run(self):
        if not self.public:
            warn('not public, not uploading')
            return
        return orig.run(self)

    The accompanied setup.py might look something like this.

    from setuptools import setup

setup(
    name='my_pypi_uploader',
    version='0.0',
    description='"safer" pypi uploader',
    py_modules=['my_pypi_uploader'],  # assuming above file is my_py_uploader.py
    entry_points={
        'distutils.commands': [
            'upload = my_pypi_uploader:upload',
        ],
    },
)

    Install that as a package into your environment and the upload command will be replaced by your version. Example run:

    $ python setup.py upload
running upload
not public, not uploading

    Try again with the public flag

    $ python setup.py upload --public
running upload
error: No dist file created in earlier command

    Which is fine, since I didn't create any dist files at all. You could of course further extend that command by rewriting the upload_file method (make a copy in your code) and change the parts to do what you want in your subclass (like injecting the private classifier there), up to you.

    You might also be wondering why the class names are in lower case (violation of pep8), this is due to legacy stuff and how the help for a given command is generated.

    $ python setup.py upload --help
...
Options for 'upload' command:

    Using a "properly" named class (e.g. SafeUpload; remember to also update the entry_point in the setup.py to point to this new class name)

    $ python setup.py upload --help
...
Options for 'SafeUpload' command:

    of course if this output is the intent, the standard class naming convention can be used instead.

    Though to be perfectly honest, you should not specify upload at all on your production, but rather do this on your build servers as part of post-push hook, so when the project is pushed (or tagged), build is done and the file is loaded onto your private servers, and then only further manual intervention (or automatic if specific tags are pushed) will then get the package up to pypi. However the above example should get you started in what you originally set out to do.

    One last thing: you can just change self.repository to your private devpi location, if the --public flag is not set. You could either override this before calling the orig.upload_file method (through your customized version), or do it in run; so rather than quitting, your code could just verify that the repository url is not the public PyPI instance. Or alternatively, manipulate the distribution metadata (i.e. the classifiers) via self.distribution.metadata (self being the upload instance). You can of course create a completely new command to play with this to your hearts content (by creating a new Command subclass, and add a new entry_point for that).