Search code examples
pythonpipconda

How to recreate Conda environment with a single command


I have to use Conda and pip together because some packages I need are only available via Conda, whereas others are only available via PyPI.

I'm following this guide carefully to avoid putting my environment in a broken state.

Note the following excerpts:

Running conda after pip has the potential to overwrite and potentially break packages installed via pip. Similarly, pip may upgrade or remove a package which a conda-installed package requires.

Creating conda packages for all additional software needed is a reliably safe method for putting together a data science environment but can be a burden if the environment involves a large number of packages which are only available on PyPI. In these cases, using pip only after all other requirements have been installed via conda is the safest practice.

Only after conda has been used to install as many packages as possible should pip be used to install any remaining software. If modifications are needed to the environment, it is best to create a new environment rather than running conda after pip.

Because of that, I frequently need to remove and recreate my Conda environment.

Here is how I do that:

# Dump the environment to a file
$ conda env export > environment.yml

# Deactivate the environment, so it becomes deletable
$ conda deactivate

# Delete the environment
$ conda env remove -n my-env

# Recreate the environment from the file
$ conda env create -f environment.yml -v

# Activate the new environment
$ conda activate my-env

Is there an easier way to do all of that with one command?

I suppose I could write a shell script, but some of the commands take an arbitrary amount of time to complete, and I don't know how to time everything correctly.

Something like conda env recreate would be ideal.


Solution

  • I solved this by writing a shell script conda_env_recreate.sh:

    #!/usr/bin/env zsh
    
    env_file='environment.yml'
    env_name='my-env'
    
    echo 'Dumping Conda environment to file.'
    conda env export --name $env_name > 'new_'$env_file
    
    echo 'Deactivating Conda environment.'
    conda deactivate
    
    echo 'Deleting Conda environment.'
    conda env remove -n $env_name
    
    echo 'Recreating Conda environment from file.'
    conda env create -f $env_file -v
    
    echo 'Reactivating Conda environment.'
    conda activate $env_name
    
    # This next step requires Kaleidoscope: https://kaleidoscope.app
    if ! cmp -s $env_file 'new_'$env_file
    then
        echo 'Comparing old and new Conda environment file.'
        ksdiff $env_file 'new_'$env_file
    fi
    

    And then I run it like this:

    source conda_env_recreate.sh