Search code examples
pythontensorflowkeraskaggle

Model training failing after 1 epoch


I am a noob in python and needed to train a model on a dataset.I found both the notebook and dataset at the same place and made appropriate changes to notebook to run the data from storage.The code fails at the training stage after 1 epoch completes with 'graph execution error'

Here is my jupyter notebook:https://github.com/Megahedron69/wasteSegregationmodel

Here is the dataset:https://www.kaggle.com/datasets/aashidutt3/waste-segregation-image-dataset

Here is the original notebook:https://www.kaggle.com/code/gpiosenka/waste-f1-score-97

enter image description here

exact error location in notebook: enter image description here

Entire error: enter image description here enter image description here


Solution

  • Thanks to @Dr. Snoopy answer.There were corrupt images in my data set so used a simple python script in my root directory to remove truncated images

    #pip install pillow
    from PIL import Image
    import os
    
    def find_truncated_images(dataset_dir):
        truncated_images = []
        for root, _, files in os.walk(dataset_dir):
            for filename in files:
                file_path = os.path.join(root, filename)
                try:
                    with Image.open(file_path) as img:
                        img.load()
                except (IOError, OSError) as e:
                    # Log the file path if it's a truncated image
                    print(f"Truncated image: {file_path}")
                    truncated_images.append(file_path)
        return truncated_images
    
    def remove_truncated_images(truncated_images):
        for file_path in truncated_images:
            try:
                os.remove(file_path)
                print(f"Removed: {file_path}")
            except OSError as e:
                print(f"Error removing {file_path}: {e}")
    
    if __name__ == "__main__":
        dataset_dir = "."  # Set the path to your dataset directory
        truncated_images = find_truncated_images(dataset_dir)
        remove_truncated_images(truncated_images)