Search code examples
pythonpipanacondavirtual-environment

best way to export anaconda environment but also include custom made packages?

I am developing a python package at work and intend to make it available to everyone, and it will come along with an environment.yml file that will set up the anaconda environment upon install so everything works out of the box.

I know I can export my anaconda like this:

conda env export > environment.yml

I have a number of packages installed in this environment that were not available using conda install, but were downloaded using pip.

From what I've read here if I have an issue with pip installed packages not showing up in my yml file, I can fix that.

My question is, I have a couple of other custom built packages that are also in this environment. They are stored on our local network, and I basically copied them to my computer and ran python setup.py install to install them.

What is the best way to distribute my new package, but include all dependencies (including the custom built packages)?

Does it make sense to include the zip files of these packages in my folder when I distribute? I haven't ran into this issue before and was hoping anyone could provide a suggestion/advice.

Thanks

Solution

  • Here are some approaches that worked for me:

    First of all create wheel files out of the custom python packages using 'python setup.py bdist_wheel' command (Build a wheel/egg and all dependencies for a python project).

    Two ways you can share the yaml file with the wheel file information:

    1. Upload the wheel file to Github, get the URL and specify the URL to wheel file under 'pip' parameter in the yaml (Specifying a url to a .whl file in a conda env .yml file)
    2. Share a zip to the target user which contains the yaml file as well as the wheel files in the same directory. You can specify the wheel file under the 'pip' parameter:

    wheel file

    conda env create -f env.yaml

    Above command would then either pip install wheel file from URL or look in the directory of the yaml for the wheel files according to approaches above