How do I install a library permanently in Colab?

In Google Colaboratory, I can install a new library using !pip install package-name. But when I open the notebook again tomorrow, I need to re-install it every time.

Is there a way to install a library permanently? No need to spend time installing every time to use?

Solution

If you want a no-authorization solution. You can use mounting with gcsfuse + service-account key embedded in your notebook. Like this:

# first install gcsfuse
%%capture
!echo "deb http://packages.cloud.google.com/apt gcsfuse-bionic main" > /etc/apt/sources.list.d/gcsfuse.list
!curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
!apt update
!apt install gcsfuse

Then get your service account credential from google cloud console and embed it in the notebook

%%writefile /key.json
{
  "type": "service_account",
  "project_id": "kora-id",
  "private_key_id": "xxxxxxx",
  "private_key": "-----BEGIN PRIVATE KEY-----\nxxxxxxx==\n-----END PRIVATE KEY-----\n",
  "client_email": "[email protected]",
  "client_id": "100380920993833371482",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/colab-7%40kora-id.iam.gserviceaccount.com"
}

Then set environment to look for this credential file

%env GOOGLE_APPLICATION_CREDENTIALS=/key.json

You must then create (or have it already) a gcs bucket. And mount it to a made-up directory.

!mkdir /content/my-bucket
!gcsfuse my-bucket /content/my-bucket

Then finally, install the library there. Like my above answer.

import sys
nb_path = '/content/my-bucket'
sys.path.insert(0, nb_path)
# Do this just once
!pip install --target=$nb_path jdc

You can now import jdc without !pip install it next time.