Search code examples
pythondownloadpytorchtorch

Unable to download saved model from online resource, pickle error


I am unable to download and use a model I saved earlier from a online-repository. Here's the code:


model = Model().double()   # Model is defined in another class
state_dict = torch.hub.load_state_dict_from_url(r'https://filebin.net/j2977ux7kts41aft/checkpoint_best.pt?t=wjbujfoo')
model.load_state_dict(state_dict)
model.eval()

Which gives me the following error:

Traceback (most recent call last):
  File "/path/file.py", line 47, in <module>
    state_dict = torch.hub.load_state_dict_from_url(r'https://filebin.net/j2977ux7kts41aft/checkpoint_best.pt?t=wjbujfoo')
  File "anaconda3/envs/torch_env/lib/python3.6/site-packages/torch/hub.py", line 466, in load_state_dict_from_url
    return torch.load(cached_file, map_location=map_location)
  File "/anaconda3/envs/torch_env/lib/python3.6/site-packages/torch/serialization.py", line 386, in load
    return _load(f, map_location, pickle_module, **pickle_load_args)
  File "anaconda3/envs/torch_env/lib/python3.6/site-packages/torch/serialization.py", line 563, in _load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '\x0a'.

The model resides in: https://filebin.net/j2977ux7kts41aft/checkpoint_best.pt?t=wjbujfoo

Note that I can perfectly download it manually, and then use torch.load(path) to load it without errors, but I need to do it from code! Could it be that the serializing when downloading from url somehow messes up the pickle encoding?

Edit: I don't have to use filebin, any online-storage that supports what I try to do will suffice.


Solution

  • The problem was indeed within the environment configuration. I created the model with PyTorch 1.0.2 and then updated to 1.2.0 in order to use torch.hub. This gave me the pickle error. After training a new model in 1.2.0, the error is now gone.

    Hope this help someone in the future :)