Is it possible to create a conda environment on a filesystem that doesn't allow symlinks (hard links or soft links)?

Question

I want to create and use a conda environment on a filesystem (azure files) that doesn't allow symlinks. Is this possible?

The environment is an Azure Container Instance with a Fileshare mounted. I'm trying to create the environment on the file share.

I've tried conda create /path/to/share -f my_env.yaml This fails with errors like Error: create soft link operation unavaliable.

What I've done that sort of woks is creating the env in the containers home space (normal filesystem allowing symlinks) and then copying across to the file share with the copy option to follow symlinks e.g:`

conda create ~/temp_env_location -f my_env.yaml  
cp -rL ~/temp_env_location /path/to/share

This sort of works but it duplicates many thousands of files even for a small environment. Is very slow and I haven't tested it enough to be sure it works well.

Any suggestions for a better approach?

Answer 1

No Softlinks

First, I'm a bit surprised that it attempts softlinking at all, since the default configuration option is allow_softlinks: false. Perhaps try setting that explicitly:

conda config --set allow_softlinks false

and see if the conda create will work by default. If not then...

Copy Only

The conda create command has a flag for what you're doing:

Package Linking and Install-time Options:
  --copy                Install all packages using copies instead of hard- or
                        soft-linking.

So you can do

conda create --copy /path/to/share -f my_env.yaml

and avoid having to do the copying manually. Unfortunately, I think this will ultimately result in the same disk usage.

Copy Always

If you will always need to use this option, then you can permanently set the always_copy option in the configuration:

conda config --set always_copy true