Search code examples
google-cloud-platformgoogle-cloud-functionskaggle

How to install Kaggle on Jupyter Notebook services in Google Cloud


I've using Google Colab for computing my Kaggle competition, nowadays I decided to take a look if it'll work faster using services on Google Cloud. I have a *.ipybn file from Google Cloud, downloaded it and try to upload it to Google Cloud instance.

I created all connection on Google Colab using this link: https://towardsdatascience.com/setting-up-kaggle-in-google-colab-ebb281b61463 and it worked fine.

Using this tutorial: https://towardsdatascience.com/how-to-use-jupyter-on-a-google-cloud-vm-5ba1b473f4c2 I started a new instance for Jupyter notebook. Uploaded a *ipybn file, I tried to install Kaggle and run my notebook, but I have usually following errors:

kaggle: command not found error ensure that your python binaries are on your path

How can I set everything to work on Google Cloud service?


Solution

  • Using this first tutorial mind about changing root directory path from content to /home/jupyter/, for example:

    import zipfile
    zip_ref = zipfile.ZipFile("/home/jupyter/Airbus_competition/input/test_v2.zip", 'r')
    zip_ref.extractall("/home/jupyter/Airbus_competition/input/test_v2")
    zip_ref.close()
    

    For problems with installing kaggle, you don't have access to root folder from Jupyter notebooks, but you can install and use Kaggle API, when you change the command from !kaggle to !~/.local/bin/kaggle, for example (commands from tutorial changed to be working on GCS):

    !mkdir ~/.kaggle
    
    
    import json
    token = {"your_TOKEN"}
    with open('/home/jupyter/.kaggle/kaggle.json', 'w') as file:
        json.dump(token, file)!cp /home/jupyter/.kaggle/kaggle.json 
    
    
    ~/.kaggle/kaggle.json
    
    
    !~/.local/bin/kaggle config set -n path -v{home/jupyter/Airbus_competition}
    
    
    !chmod 600 /home/jupyter/.kaggle/kaggle.json
    
    
    !~/.local/bin/kaggle competitions download -c airbus-ship-detection -p /home/jupyter/Airbus_competition/input --force