OSError: [Errno 22] Invalid argument | _pickle.UnpicklingError: pickle data was truncated

Question

When trying to train a ResNet I get this error. Any help as to why this happens would be appreciated. This happens when I try to iterate through the Dataloader:

File "C:\Users\JCout\AppData\Local\Temp/ipykernel_2540/2174299330.py", line 1, in <module>
    runfile('C:/Users/JCout/Documents/GitHub/Hybrid_resnet/transfer_learning.py', wdir='C:/Users/JCout/Documents/GitHub/Hybrid_resnet')

  File "C:\Users\JCout\anaconda3\lib\site-packages\debugpy\_vendored\pydevd\_pydev_bundle\pydev_umd.py", line 167, in runfile
    execfile(filename, namespace)

  File "C:\Users\JCout\anaconda3\lib\site-packages\debugpy\_vendored\pydevd\_pydev_imps\_pydev_execfile.py", line 25, in execfile
    exec(compile(contents + "\n", file, 'exec'), glob, loc)

  File "C:/Users/JCout/Documents/GitHub/Hybrid_resnet/transfer_learning.py", line 24, in <module>
    model, train_loss, test_loss = train.train_model(training, testing,

  File "C:\Users\JCout\Documents\GitHub\Hybrid_resnet\train.py", line 70, in train_model
    train_stats = train.train_step(model, criterion, optimizer, train_loader)

  File "C:\Users\JCout\Documents\GitHub\Hybrid_resnet\train.py", line 121, in train_step
    for i, (x_imgs, labels) in enumerate(train_loader):

  File "C:\Users\JCout\anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 359, in __iter__
    return self._get_iterator()

  File "C:\Users\JCout\anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 305, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)

  File "C:\Users\JCout\anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 918, in __init__
    w.start()

  File "C:\Users\JCout\anaconda3\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)

  File "C:\Users\JCout\anaconda3\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)

  File "C:\Users\JCout\anaconda3\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)

  File "C:\Users\JCout\anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)

  File "C:\Users\JCout\anaconda3\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)

OSError: [Errno 22] Invalid argument

I'm also getting this traceback after the error which I find a bit weird since I'm not using pickle at all. The data consists of 2 .tif files and 2 .mat files for the data/targets.

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\JCout\anaconda3\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\JCout\anaconda3\lib\multiprocessing\spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
_pickle.UnpicklingError: pickle data was truncated
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\JCout\anaconda3\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\JCout\anaconda3\lib\multiprocessing\spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
_pickle.UnpicklingError: pickle data was truncated

Answer 1

Not entirely sure what was happening, but I had

pin_memory = True
num_workers = 4

within the DataLoader. Removing these removed the error.