Is there a way to efficiently extract all PILImages from fastai.data.core.Dataset object?

Question

I was trying to rewrite my code from fastai1 to fastai2 due to GPU incompatibility, and I run into a problem with extracting images:

fastai1 (extract all images from train_ds):

data.train_ds.x

I have 100,000 images as input, and my list comprehension technique doesn't work (the thread was killed every time I try to run it):

[x[0] for x in data.train_ds]

Is there a better way to extract all the images?

Answer 1

[x[0] for x in data.train_ds]

was loading all the images into memory at once, the correct way to approach the problem is load them in dataset creation.

Solutions for this problem varies what one want to do with these images: Since I need these images to create a custom fastai dataset, I decided to load the images only in the dataset and pass data.train_ds to the dataset.

The code is

CustomDataset(data.train_ds,...)

In CustomDataset (probably in getitem()) use:

self.images[i][0]