Search code examples
pythondeep-learningpytorchremote-debugginghelper

Pytorch DataLoader doesn't work with remote interpreter


I have the following error.

Expected: /home/ubuntu/.pycharm_helpers/pydev/pydevd_attach_to_process/attach_linux_amd64.so to exist.

And Here is the code:

import torch_geometric.transforms as T
category = 'Airplane'
path = osp.join(osp.dirname(osp.realpath(__file__)), '..', 'data', 'ShapeNet')
transform = T.Compose([
    T.RandomTranslate(0.01),
    T.RandomRotate(15, axis=0),
    T.RandomRotate(15, axis=1),
    T.RandomRotate(15, axis=2)
])
pre_transform = T.NormalizeScale()
train_dataset = ShapeNet(path, category, split='trainval', transform=transform,
                         pre_transform=pre_transform)
test_dataset = ShapeNet(path, category, split='test',
                        pre_transform=pre_transform)
train_loader = DataLoader(train_dataset, batch_size=12, shuffle=True, num_workers=6)
test_loader = DataLoader(test_dataset, batch_size=12, shuffle=False,
                         num_workers=6)

When I'm trying sample from dataset using the dataloader, the debugger crashes and returns this erros.

Have tried deleting remote helpers, but it didn't solve my problem. My local machine in running on windows 10 and the remote one is running on Ubuntu 18.04


Solution

  • Answering to myself here,

    I've tried to follow this : https://intellij-support.jetbrains.com/hc/en-us/community/posts/360006791159--pycharm-helpers-pydev-pydevd-attach-to-process-attach-linux-amd64-so-undefined-symbol-AttachDebuggerTracing

    But it has proven inefficient.

    On the contrary, deleting the parameter 'num_workers' from the Dataloader solved this issue. So this is an easy workaround.

    However, I haven't wrap my head around the problem yet. It's probably that multiprocessing is buggy. But if anyone has any insight of what's going on. I'll gladly take it.