Search code examples
pythongoogle-colaboratorykaggle

Can't find kaggle.json file in google colab

I'm trying to download the kaggle imagenet object localization challenge data into google colab so that I can use it to train my model. Kaggle uses an API for easy and fast access to their datasets. (https://github.com/Kaggle/kaggle-api) However, when calling the command "kaggle competitions download -c imagenet-object-localization-challenge" in google colab, it can't find the kaggle.json file which contains my username and api-key.

I haven't had this problem on my mac when running a jupyter notebook, but since I want to use google's gpu for my model, I started using google colab. Because the kaggle API expects the username and api-key to be in a kaggle.json file located in a .kaggle directory, I first created the directory .kaggle and then the file kaggle.json, into which I wrote my username and api-key (The example below doesn't display my username and api-key). I then tried to configure the path to my json file for kaggle to use when calling the kaggle download command.

!pip install kaggle

!mkdir .kaggle
!touch .kaggle/kaggle.json

api_token = {"username":"username","key":"api-key"}

import json
import zipfile
import os
with open('/content/.kaggle/kaggle.json', 'w') as file:
    json.dump(api_token, file)

!chmod 600 /content/.kaggle/kaggle.json
!kaggle config path -p /content

However, when running the last command, I got the error:

IOError: Could not find kaggle.json. Make sure it's located in /root/.kaggle. Or use the environment method.

My goal was to use the following commands to get the dataset from kaggle:

!kaggle competitions download -c imagenet-object-localization-challenge
os.chdir('/content/competitions/imagenet-object-localization-challenge')
for file in os.listdir():
    zip_ref = zipfile.ZipFile(file, 'r')
    zip_ref.extractall()
    zip_ref.close()

I don't understand why the kaggle API can't find my json file. How can I use the API in google colab?

Solution

  • As the error said, you need to put kaggle.json in the right place.

    Try:

    !mv .kaggle /root/

    Then run your code again.