FileNotFoundError: [Errno 2] No such file or directory: '2007_009096.jpg'

Question

I recently made this class :

class RealPhotosDataset(Dataset):
    def __init__(self, directory):
        self.files = os.listdir(directory)

    def __getitem__(self, index):
        print(self.files[index])
        img = Image.open(self.files[index]).convert('RGB')
        return T.ToTensor()(img)
    
    def __len__(self):
      return len(self.files)

data_set = RealPhotosDataset("targetdir/JPEGImages/")

train_len = int(len(data_set)*0.7)
train_set, test_set = random_split(data_set, [train_len, len(data_set) - train_len])

kwargs = {'num_workers': 1, 'pin_memory': True}
train_loader = DataLoader(train_set, batch_size=1, shuffle=True, **kwargs)
test_loader = DataLoader(test_set, batch_size=64, shuffle=True, **kwargs)

I loaded my dataset in colab :

I have a for loop, i.e

for x, y in test_loader:
   ...

which create a challenge fixing the error.

FileNotFoundError: Caught FileNotFoundError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 198, in _worker_loop
    data = fetcher.fetch(index)
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataset.py", line 272, in __getitem__
    return self.dataset[self.indices[idx]]
  File "<ipython-input-58-be7e605a57d0>", line 7, in __getitem__
    img = Image.open(self.files[index]).convert('RGB')
  File "/usr/local/lib/python3.6/dist-packages/PIL/Image.py", line 2809, in open
    fp = builtins.open(filename, "rb")
FileNotFoundError: [Errno 2] No such file or directory: '2007_009096.jpg'

I have not idea how to fix this error. 2007_009096.jpg is really an existing file in targetdir/JPEGImages. How can I fix that error

Answer 1

Replace

self.files = os.listdir(directory)

by

self.files = [os.path.join(directory, f) for f in os.listdir(directory)]

to correctly register the full path of the files.