Search code examples
pythonserveranacondaconda

Conda installation in an ssh server

I want to install Miniconda and create an environment where I can install specific packages. For installing Miniconda I am following these instructions. In the server, there is home partition but also local and data partition. The home directory has a strict storage limitation so I would like to install everything in /local/$USER instead of /home/$USER.

So I am doing the Miniconda installation on the /local/$USER and then I create an environment with a specific Python version using:

conda create -y -name my_username python=3.12

That command, unfortunately, automatically installs the environment in the /home/$USER directory. Moreover, I found an ugly command that can enforce the environment installation to be made in a specific directory:

conda create -p /local/$USER/conda_envs/my_conda_env

Then I need to do sth like:

source activate /local/$USER/conda_envs/my_conda_env

And install all my packages. During the installation though I noticed that all these packages are still installed in the/home/$USER/ directory and I am still ran out of storage. Even when I do all the Miniconda installations on the local/$USER/, if then I write conda info, I have figured out that the user config file : /home/$USER/.condarc and cache files are still in the home directory (/home/$USER/.conda/envs).

How can I properly deal with this? I guess there was an old conda installation with instructions somewhere to install environments on the /home/$USER folder, how can I make /local/$USER the default one?

The steps that I have followed for installing miniconda are the following:

cd /local/$USER
mkdir -p miniconda3/
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda3/miniconda.sh
bash miniconda3/miniconda.sh -b -u -p miniconda3/
rm miniconda3/miniconda.sh

Is there sth in the installation that could lead to /home/ directory? I have inspect the miniconda.sh and there is this PREFIX="${HOME:-/opt}/miniconda3" but during the installation, there is a message that says PREFIX="local/$USER/miniconda3"

Solution

  • I have met similar issues before. I will try to restore the process according to your configuration:

    1. Create .condarc (use vim or nano, up to you):
    nano /local/$USER/.condarc
    1. Add the following content to the file:
    envs_dirs:
- /local/$USER/.conda/envs
pkgs_dirs:
- /local/$USER/.conda/pkgs

    1. Save and close the file.

    2. Edit the .bashrc file:

    nano /local/$USER/.bashrc
    1. Add the following lines:
    export CONDA_ENVS_PATH="/local/$USER/.conda/envs"
export CONDA_PKGS_DIRS="/local/$USER/.conda/pkgs"
    1. Save the file and run:
    source /local/$USER/.bashrc
    1. Reinitialise Conda:
    /local/$USER/miniconda3/bin/conda init
    1. Finally, check that everything is OK:
    conda info

    Make sure the path shown in the output points to /local/$USER/ and not /home/$USER/. That means it's OK. When you create new environments, they should be installed by default in the /local/$USER/.conda/envs directory.