Search code examples
pythongoogle-colaboratorypickle

Error when trying to load pickle files in Google Colab


I am using Google Colab to try to load pickle files to an array. However, it doesn't find the path to my data. Currently my data is set as:

/content/drive/My Drive/My Folder/Data/Storage/My Pickle Files/

And my drive is mounted to the path:

drive.mount('/content/drive')

And I have an array of pickle files, that I am trying to load them into the array, It doesn't seem to find the correct path to load those files, I have tried using the code described below:

p_files = [
        "content/drive/My Drive/My Folder/Data/Storage/My Pickle Files/pickles_1.pkl",
        "content/drive/My Drive/My Folder/Data/Storage/My Pickle Files/pickles_2.pkl",
        "content/drive/My Drive/My Folder/Data/Storage/My Pickle Files/pickles_3.pkl",
        "content/drive/My Drive/My Folder/Data/Storage/My Pickle Files/pickles_4.pkl",
        "content/drive/My Drive/My Folder/Data/Storage/My Pickle Files/pickles_5.pkl",
        "content/drive/My Drive/My Folder/Data/Storage/My Pickle Files/pickles_6.pkl"   
    ]

loaded_pickles = []

for p in p_files:
    with open(p, 'rb') as file:
        loaded_pickles.append(pickle.load(file))

I get this error:

FileNotFoundError: [Errno 2] No such file or directory: 'content/drive/My Drive/My Folder/Data/Storage/My Pickle Files/pickles_1.pkl'

Solution

  • You have to specify that the pickle files are in your Drive.

    E.g. "content/drive/My Drive/My Folder/Storage/My Pickle Files/pickles_1.pkl"

    Otherwise it assumes that the files are local.

    Alternatively, you could port the files over to local using shutil or similar, but I don't see why you would.

    Response to OP's comment: perhaps you could try porting over the files to local using something like:

    import shutil
    for file in p_files:
        filename = file[62:]
        shutil.copy(file, filename)
    new_p_files = [f[62:] for f in p_files]
    

    Then load new_p_files using your original method?