Why does flake8 give different results for the same command locally and within tox?

I have a Python project and I use flake8 to lint my code.

Locally, there is a warning rightly raised by flake8:

$ flake8 scripts src tests --ignore=W503,E501
src/projectname/workers/utils.py:22:20: W291 trailing whitespace

However, the same command in tox does not raise any warning:

$ tox -e flake8

gets the following output:

flake8 develop-inst-noop: /home/username/Projects/projectname
flake8 installed: alembic==0.9.8, ..., zipp==0.5.2
flake8 runtests: PYTHONHASHSEED='2190899390'
flake8 runtests: commands[0] | flake8 scripts src tests --ignore=W503,E501
________________________________ summary _________________________________
  flake8: commands succeeded
  congratulations :)

This is the content of my tox.ini file:

[tox]
envlist = flake8,py36

[testenv]
changedir = {toxworkdir}/{envname}
usedevelop = True
install_command = pip install {opts} {packages}
deps =
    py36: pytest-cov
    py36: pytest
    flake8: flake8
setenv =
    COVERAGE_FILE = {toxinidir}/.coverage.{envname}
commands =
    py36: pytest {toxinidir}/tests --cov=projectname {posargs}
    flake8: flake8 scripts src tests --ignore=W503,E501
    ;       E501: line too long
    ;       W503: line break before binary operator

I checked, and in both experiments, I have flake8==3.7.9
Why does flake8 and tox not return any error code in this case when they should?

Solution

your tox.ini has:

changedir = {toxworkdir}/{envname}

this means that when you run flake8 in tox, it's linting (non-existent) .tox/flake8/scripts / .tox/flake8/src / .tox/flake8/tests and so you don't see an error (the 3.7.x behaviour is to silently ignore non-existent things)

note that in flake8 3.8 (at the time of writing, unreleased) this will yet-again become an error (E902):

$ flake8 does-not-exist
does-not-exist:0:1: E902 FileNotFoundError: [Errno 2] No such file or directory: 'does-not-exist'

disclaimer: I'm the current maintainer of flake8, and one of the maintainers of tox