I've inherited a dataset stored in a pickle file, which was to the best of my knowledge, created in python 3.
I am trying to open the pickle file in python using the following code:
f = open('file.pkl', 'rb')
cl = pickle.loads(f)
When I do so, I get the following error:
TypeError: a bytes-like object is required, not '_io.BufferedReader'
A similar error is thrown when I try to read it in with only an 'r'
argument.
I can see that the pickel file has been created with joblib, which I have never used before. I can't find any documentation saying if joblib pickel files need to be opened a different way.
Here is the code to create the pickle file:
joblib.dump(list_of_dataframes, 'file.pkl', compress=3)
Any help is greatly appreciated!
EDIT
This also won't open when I try to load directly from joblib
:
with open(pickle_file) as f:
data = joblib.load(f)
Which throws the following error:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
I found the answer, and it wasn't what I expected! (Thank you @JohnGordon for earlier help.) My file was corrupted because it was too large to do a direct pull from GitHub. To fix this, I used Git LFS and did a git lfs pull
instead of a git pull
. Now unpickling works without errors.