How does pylint use easy_install at runtime?

Sorry this is a long question. See the sentence in bold at the bottom for the TL;DR version.

I've spent many hours trying to track down a problem where pylint sometimes doesn't report all the errors in a module. Note that it does find some errors (e.g. long lines), just not all of them (e.g. missing docstrings).

I'm running pylint 1.7.2 on Ubuntu 16.04. (The version available from apt was 1.5.2 but installing via pip gives 1.7.2.)

We typically run pylint from tox, with a tox.ini that looks something like this (this is a cut-down version):

[tox]
envlist = py35
[testenv]
setenv =
    MODULE_NAME=our_module
ignore_errors = True
deps =
    -r../requirements.txt
whitelist_externals = bash
commands =
    pip install --editable=file:///{toxinidir}/../our_other_module
    pip install -e .
    bash -c \'set -o pipefail; pylint --rcfile=../linting/pylint.cfg our_module | tee pylint.log\'

Amongst other things, the ../requirements.txt file contains a line for pylint==1.7.2.

The behaviour is like this:

[wrong] When the line that imports our_other_module is present, pylint appears to complete successfully and not report any warnings, even though there are errors in the our_module code that it should pick up.
[correct] When that line is commented out, pylint generates the expected warnings.

As part of tracking this down I took two copies of the .tox folder with and without the module import, naming them .tox-no-errors-reported and .tox-with-errors-reported respectively.

So now, even without sourcing their respective tox virtualenvs, I can do the following:

$ .tox-no-errors-reported/py35/bin/pylint --rcfile=../linting/pylint.cfg our_module -- reports no linting warnings
$ .tox-with-errors-reported/py35/bin/pylint --rcfile=../linting/pylint.cfg our_module -- reports the expected linting warnings

(where I just changed the pylint script's #! line in each case to reference the python3.5 inside that specific .tox directory instead of the unrenamed .tox)

By diffing .tox-no-errors-reported and .tox-with-errors-reported, I've found that they are very similar. But I can make the "no errors" version start to report errors by removing the path to our_other_module from .tox-no-errors-reported/py35/lib/python3.5/site-packages/easy-install.pth.

So my question is why is pylint using easy_install at runtime, and what is it picking up from our other component that is causing it to fail to report some errors.

As I understand it, pylint has dependencies on astroid and logilab-common, but including these in the requirements.txt doesn't make any difference.

Solution

One possible reason for the surprising pylint behavior is the --editable option.

it creates a special .egg-link file in the deployment directory, that links to your project’s source code. And, ..., it will also update the easy-install.pth file to include your project’s source code

The pth file will then affect the sys.path which affects the module import logic of astroid and it is deeply buried in the call stack of pylint.expand_files via pylint.utils.expand_modules. Also pylint identifies the module part and function names in the AST using astroid.modutils.get_module_part.

To test the theory, you can try calling some of the affected astroid functions manually:

import sys, astroid
print(sys.path)
print(astroid.modutils.get_module_part('your_package.sub_package.module'))
astroid.modutils.file_from_modpath(['your_package', 'sub_package', 'module'])