Search code examples
python-3.xpipcondaconda-forge

Conda environment way larger due to additional dependencies

following content of environment.yml:

name: ong_env
channels:
  - conda-forge
  - defaults
dependencies:
  - appdirs
  - atomicwrites
  - attrs
  - autopep8
  - black
  - ca-certificates
  - certifi
  - click
  - colorama
  - coverage
  - execnet
  - flake8
  - flake8-polyfill
  - hypothesis
  - importlib-metadata
  - importlib_metadata
  - iniconfig
  - mccabe
  - more-itertools
  - mypy
  - mypy_extensions
  - openssl
  - packaging
  - pathspec
  - pep8-naming
  - pip
  - pluggy
  - psutil
  - py
  - pycodestyle
  - pyflakes
  - pyparsing
  - pytest
  - pytest-cov
  - pytest-forked
  - pytest-html
  - pytest-metadata
  - pytest-xdist
  - python
  - pywin32
  - regex
  - ruamel.yaml
  - ruamel.yaml.clib
  - setuptools
  - snakeviz
  - sortedcontainers
  - sqlite
  - toml
  - tomli
  - tornado
  - typed-ast
  - typing_extensions
  - vc
  - vs2015_runtime
  - wheel
  - wincertstore
  - zipp
  - zlib
  - antlr4-python3-runtime
  - chardet
  - charset-normalizer
  - decorator
  - deepdiff
  - idna
  - importlib-resources
  - jsonpath-ng
  - jsonschema
  - markdown
  - multipledispatch
  - ordered-set
  - ply
  - pycparser
  - pyrsistent
  - pythonnet
  - requests
  - six
  - types-markdown
  - untangle
  - urllib3
  - xlsxwriter
  - semi-ate-stdf
  - pip:
    - api4jenkins
    - jsonmerge
    - json-checker
    - logging-logger
    - nesteddict

results in a new conda environment (using conda env create -f environment.yml --prefix .\ong_env) of about 700 MB. If I just move the line - semi-ate-stdf one line down (to use the pip version of semi-ate instead of the conda-forge one), the resulting environment is 213 MB in size! I am clear about the reason - conda-forge introduces some new dependencies that are huge (like clang and qt5).
The question is why is conda-forge requiring so many additional packages that pip does not need. Mark that both tools (conda as well as pip) install version 0.1.26 of semi-ate.
Or is there a way to tell conda to create an environment that is "as small as possible", although I would think that already is the default, isn't it?

As we store our virtual environment in SVN and many users need to check it out of the repository the increased size - by a factor of more than 3 - makes a huge difference to us.

Solution

  • The latest PyPI version (v0.1.26) dropped a PyQt requirement, but the Conda recipe didn't correctly update this metadata. Since PyQt entails Qt, which is a huge framework, that is likely where most of the heft is originating and would be differential between the two environments described.

    A pull request has been filed to correct this, so eventually you should be able to get a more comparable environment size out of the Conda version once that merges and the build distributes.