Why does my ImageDataGenerator not detect any images?

Question

I am trying to augment my image data, however for some reason it does not detect my image paths. I am augmenting for images in 3 classes: dirty, vehicle and clean and then transfer the augmented images to another folder with the same structure. I have a suitable folder structure as well.

import os
from tensorflow.keras.preprocessing.image import ImageDataGenerator

# Set the paths to your dataset folders
data_dir = "/home/intern/Desktop/automated-py-scripts/real_data"
new_augment_dir = "/home/intern/Desktop/automated-py-scripts/augmented"
train_dir = os.path.join(data_dir, "train")

# Define the augmentation parameters
augmentation_params = {
    "rotation_range": 15,
    "width_shift_range": 0.1,
    "height_shift_range": 0.1,
    "shear_range": 0.1,
    # "zoom_range": 0.1,
    "horizontal_flip": True,
    "fill_mode": "nearest"
}

# Create an ImageDataGenerator instance with augmentation parameters
datagen = ImageDataGenerator(**augmentation_params)

# Get the list of class folders
class_folders = ['clean', 'dirty', 'vehicle']

# Augment images in each class folder
for folder in class_folders:
    class_dir = os.path.join(train_dir, folder)
    print(class_dir)
    save_dir = os.path.join(new_augment_dir, folder)

    # Create the save directory if it doesn't exist
    os.makedirs(save_dir, exist_ok=True)

    # Create a generator for the images in the class folder
    generator = datagen.flow_from_directory(
        class_dir,
        target_size=(224, 224),  # Resize the images to your desired size
        batch_size=1,
        class_mode=None,
        save_to_dir=save_dir,
        save_format="jpg",
        shuffle=True
    )

    # Generate augmented images and save them to the save directory
    num_images = len(os.listdir(class_dir))
    num_augmented_images = 3  # Number of augmented images to generate per original image

    for _ in range(num_images * num_augmented_images):
        images = next(generator)

The issue I am facing rises from the class_dir variable, it prints out the correct path but there is no detection.

/home/intern/Desktop/automated-py-scripts/real_data/train/clean
Found 0 images belonging to 0 classes.
/home/intern/Desktop/automated-py-scripts/real_data/train/dirty
Found 0 images belonging to 0 classes.
/home/intern/Desktop/automated-py-scripts/real_data/train/vehicle
Found 0 images belonging to 0 classes.

Do I need to change my folder structure again?

Answer 1

As pointed out by Dr Snoopy the way the ImageDataGenerator works is that you specify a source directory that contains sub directories which are the classes. It then goes through each class directory and processes the file in that class directory. The easiest solution to your problem is that in each of your 3 class directories create an empty sub directory with the same name as the class directory. For example within dirty class directory create a sub directory also called dirty. Then move all the files in the class directory to the sub directory.