Search code examples
pythonenumspytesttravis-cipipenv

pytest: `foo is Enum.FOO` tests True locally, False on travis-ci


Halp! I'm having weird issues with checking equality for Enums. Locally my tests pass, but on Travis they fail. Here's a sample of a test failure from Travis:

_____________________________ test_extract_object ______________________________
    def test_extract_object():
        """Test UmlCreator.extract_object()."""
        creator = UmlCreator(".")
        with pytest.raises(ValueError):
            creator.extract_object(["no", "klass", "included"])
        obj = creator.extract_object(
            [
                "public",
                "static",
                "class",
                "StaticKlass",
                ":",
                "ICanBeImplemented,",
                "IComparable,",
                "IEquatable<Klass>",
            ]
        )
        assert obj.__class__.__name__ == "UmlClass"
>       assert obj.access is Access.PUBLIC
E       assert Access("public") is Access("public")
E        +  where Access("public") = UmlClass(['StaticKlass', ':', 'ICanBeImplemented,', 'IComparable,', 'IEquatable<Klass>'], **{'nsp': None, 'access': Access("public"), 'attrs': [], 'modifiers': [Modifier("static")], 'repo_url': None}).access
E        +  and   Access("public") = Access.PUBLIC
tests/test_creator.py:44: AssertionError

If I change the offending line to assert obj.access.value == Access.PUBLIC.value it passes on Travis as well, but I'm stumped as to why this fails in one place and passes in another with basically the same setup, and it of course also affects non-test code, meaning that everywhere I use foo is Enum.VAL in my code, tests for those methods fail as well:

tests/test_creator.py:44: AssertionError
_________________________ test_uml_class_display_name __________________________
    def test_uml_class_display_name():
        """Test UmlClass.display_name()."""
        klass = UmlClass(["Classy"])
        assert klass.display_name() == "Classy"
        abstract_klass = UmlClass(["Classy"], modifiers=[Modifier.ABSTRACT])
>       assert abstract_klass.is_abstract()
E       assert False

Local

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.4 LTS
Release:        18.04
Codename:       bionic
$ pip list
Package            Version
------------------ --------
appdirs            1.4.3
astroid            2.3.3
attrs              19.3.0
bandit             1.6.2
black              19.10b0
Click              7.0
coverage           5.0.3
gitdb2             2.0.6
GitPython          3.0.7
importlib-metadata 1.5.0
isort              4.3.21
lazy-object-proxy  1.4.3
mccabe             0.6.1
more-itertools     8.2.0
packaging          20.1
pathspec           0.7.0
pbr                5.4.4
pip                20.0.2
pluggy             0.13.1
py                 1.8.1
pylint             2.4.4
pyparsing          2.4.6
pytest             5.3.5
pytest-cov         2.8.1
PyYAML             5.3
regex              2020.1.8
rope               0.16.0
setuptools         45.1.0
six                1.14.0
smmap2             2.0.5
stevedore          1.32.0
toml               0.10.0
typed-ast          1.4.1
wcwidth            0.1.8
wheel              0.34.2
wrapt              1.11.2
zipp               2.2.0
$ pipenv run test
===== test session starts =====
platform linux -- Python 3.6.9, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /mnt/c/Users/thy/source/repos/uml.cs, inifile: pytest.ini
plugins: cov-2.8.1
collected 41 items

Travis

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.3 LTS
Release:    18.04
Codename:   bionic
$ pip list
Package            Version
------------------ ----------
appdirs            1.4.3
astroid            2.3.3
atomicwrites       1.3.0    !
attrs              19.3.0
bandit             1.6.2
black              19.10b0
certifi            2019.6.16 !
Click              7.0
coverage           5.0.3
gitdb2             2.0.6
GitPython          3.0.7
importlib-metadata 1.5.0
isort              4.3.21
lazy-object-proxy  1.4.3
mccabe             0.6.1
mock               3.0.5 !
more-itertools     8.2.0
nose               1.3.7 !
numpy              1.18.0 !
packaging          20.1
pathspec           0.7.0
pbr                5.4.4
pip                19.3.1
pipenv             2018.11.26
pluggy             0.13.1
py                 1.8.1
pylint             2.4.4
pyparsing          2.4.6
pytest             5.3.5
pytest-cov         2.8.1
PyYAML             5.3
regex              2020.1.8
rope               0.16.0
setuptools         42.0.2
six                1.14.0
smmap2             2.0.5
stevedore          1.32.0
toml               0.10.0
typed-ast          1.4.1
virtualenv         16.6.1
virtualenv-clone   0.5.3
wcwidth            0.1.8
wheel              0.33.6
wrapt              1.11.2
zipp               2.2.0
python -m pytest
===== test session starts =====
platform linux -- Python 3.6.9, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /home/travis/build/kthy/uml.cs, inifile: pytest.ini
plugins: cov-2.8.1
collected 41 items

For easier comparison, here's a table of the differences:

| Local (WSL)        | Travis                 |
| ------------------ | ---------------------- |
| Ubuntu 18.04.4 LTS | Ubuntu 18.04.3 LTS     |
|                    | atomicwrites-1.3.0     |
|                    | certifi-2019.6.16      |
|                    | mock-3.0.5             |
|                    | nose-1.3.7             |
|                    | numpy-1.18.0           |
| pip-20.0.2         | pip-19.3.1             |
|                    | pipenv-2018.11.26      |
| setuptools-45.1.0  | setuptools-42.0.2      |
|                    | virtualenv-16.6.1      |
|                    | virtualenv-clone-0.5.3 |
| wheel-0.34.2       | wheel-0.33.6           |

Full build log from Travis can be seen e.g. here.


Solution

  • So, as pointed out by Ethan the problem was my imports. For whatever reason I was importing the Enums like

    try:
      from features import Access
    except (ImportError, ModuleNotFoundError):
      from umldotcs.features import Access
    

    - I can't remember why, but probably because I had had some problems with imports before. Cargo cult programming strikes again! 😉 Running pytest locally used the second import, whereas Travis for some reason chose the first, triggering Issue30545 which is user error, not a python bug. Making sure all Enums were imported in the same way resolved the problem.