Search code examples
pythonversioningpypi

Releasing a non-backwards compatible library on PyPI?

I have a library on PyPI called foobar and it's currently at version 1.2.0 (using semantic versioning).

The next version doesn't preserve API compatibility with versions 1.x, so I'll release it as 2.0.0.

What is the best practice to publish this new version to PyPI, so that clients which are using the 1.x versions don't accidentally upgrade to 2.0.0 and break their code? (I'm assuming that there are people who didn't enforce a version dependency like >=1.0.0, <2.0.0 in their code).

Would it be better to create a completely new package called foobar2 on PyPI and push the new version there? How do other projects handle this?

Solution

  • I assert: API changes normally fall into two categories.

    1. New API B replaces A but A can entirely be implemented using new API B. Therefore it is feasible to maintain the old API simultaneously with the new API.

      This could be as simple as a new API being moved to tidy or rationalize your module, or more complex such as a conversion of args to kwargs or whatever.

    2. New API replaces old API but cannot implement it for whatever technical reason.

    These are your options for categories IMO. Which one you take will depend a lot on what changes you are making and how much you a) care about or b) are in contact and can talk to your users (i.e. you can get away with a few unannounced breakages if it's just a few people on your team who you can subsequently help fix their issues).

    1. Provide both old and new in your new version.

    Implement the old API using the new one but mark it as deprecated using the warnings module. You give people notice to convert and you can remove the old API at some point in the future.

    This is best practice for API changes of the first type. It keeps everyone on the same stream and allows you to tidy up at some point in the future.

    2. Warn, then introduce new API.

    If you are in situation 2 or situation 1 but can't justify the resource to implement old using new, then you can easily release a version 1.2.1 that uses the warnings module to warn users that you are about to add a new version that will break their codez, and that they should quickly peg the version in their requirements.txt.

    Say when you're going to release version 2.0, and then you've warned them.

    But this is only really fair if it's not too much effort to migrate from 1.2.0 to 2.0 for your users.

    3. Add a completely new package.

    If there are profound differences, and it would be a right pain for your users to update their code to the point that they would essentially need to rewrite it, then you shouldn't be afraid of just using a completely new package. We all make mistakes, and no one in the Python community is not aware of that given the differences between Python 2 and Python 3 :-). unittest2 is also one such example from a while back.

    What will people expect.

    Personally, if I had automatic upgrades occurring on a system I cared about and if I didn't peg the versions to upgrade only maintenance releases, I would consider it my fault if someone released a major upgrade that my system automatically took but then stopped working because of it.

    That gives you the moral highground IMO, but that isn't much consolation for your lazy users (or worse, customers) who didn't check and got burnt.

    What other people do

    • paramiko kept API back-compatibility on 1.x to 2.0
    • beautifulsoup change name on PyPI (from BeautifulSoup to bs4)
    • django tends to deprecate features and remove them in later feature releases, and in general I would never upgrade a django install I cared about from 1.X to 1.(X+1) without testing it first. (I consider this to be the best practice, like a lot of things the django folk do.)

    So the summary is: there is a mix, and it really is up to you. However the only completely safe ways to avoid user self-inflicted issues is to keep back-compatibility entirely or create a new package, as BeautifulSoup did.