Search code examples
pythonpython-2.7pipsetuptoolstransitive-dependency

Detect Python transitive dependency issues at install time?


I recently learned that pip does not resolve transitive dependencies in any sane fashion. This means after installing a package, some of its dependencies' dependencies may not actually be satisfied.

Is there a way to programmatically check that all installed packages' dependencies are actually satisfied after installing?

Specifically, I would like to be able to create a virtualenv, install some packages into it, and then verify that all installed packages actually have the required dependencies.


Edit: Here is an example to better illustrate the problem. Suppose package A depends on packages B and C, both of which depend on package D, possibly with different version ranges. When installing A, pip will arbitrarily choose one of the specified version ranges for D to satisfy. For example, B's dependency on D might be satisfied, but C's required version for D may not be satisfied. I want to detect whether such problems exist.


Solution

  • This answer shows how to get all locally installed packages. This one shows a way to verify that a given package is installed, which also recursively checks the package's dependencies. Combining these:

    import pip
    import pkg_resources
    
    pkg_resources.require(str(dep.as_requirement())
                          for dep in pip.get_installed_distributions())
    

    This will raise a ResolutionError if any of the installed packages has an unsatisfied requirement.


    Aside: This seems to be what the new pip check command is intended to do, but it doesn't catch some dependency issues that the above script catches, such as extras-related issues.


    Update: Recent versions of pip do not provide a get_installed_distributions() function. Per https://github.com/pypa/pip/issues/5243, pkg_resources.working_set can be used to get the dependency list:

    import pkg_resources
    
    pkg_resources.require(str(dep.as_requirement())
                          for dep in pkg_resources.working_set)