Keras for creating CNN - Size of array double number of training images

Question

I'm creating a CNN that determines whether an image scanned is a cat or dog. I have one folder with images of both cats and dogs. The cats have "cat." before the image # and dogs have "dog."; for example, an image of a cat may be "cat.403.jpg". In total there are around 20,000-ish images. However, when I run my code below, my array stores 40,000-ish values, nearly double the amount. I don't understand what's going on. Here's the code:

DATADIR = "C:/Users/me/Jupyter Codes/dogs-vs-cats/train/train"
CATEGORIES = ['dog', 'cat']

IMG_SIZE = 50 
training_data = []
cat_img_array = []
dog_img_array = []

def create_training_data():
    for category in CATEGORIES:
        train_path = os.path.join(DATADIR)   # path to train folder
        class_num = CATEGORIES.index(category)
        for img in os.listdir(train_path):
            animal = img.split('.')[0]
            if animal == 'cat':
                cat_img_array = cv2.imread(os.path.join(train_path, img), cv2.IMREAD_GRAYSCALE)  # convert to grayscale bc RGB > gray; plus not essential
                cv2.resize(cat_img_array, (IMG_SIZE, IMG_SIZE))
                training_data.append([cat_img_array, class_num])
                
            elif animal == 'dog':
                dog_img_array = cv2.imread(os.path.join(train_path, img), cv2.IMREAD_GRAYSCALE)
                cv2.resize(dog_img_array, (IMG_SIZE, IMG_SIZE))
                training_data.append([dog_img_array, class_num])

create_training_data()
print(len(training_data))

This returns 41900 images

Then when I run this code:

# pack data into variables before fed into CNN
X = []
Y = []

for features, label in training_data:
    X.append(features)
    Y.append(label)

    
X = np.array(X).reshape(-1, IMG_SIZE, IMG_SIZE, 1) # -1 -> make array number based on data; 1 -> grayscale (3 if RGB)
Y = np.array(Y).reshape(-1, IMG_SIZE, IMG_SIZE, 1)

I get these error messages:

VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray. X = np.array(X).reshape(-1, IMG_SIZE, IMG_SIZE, 1) # -1 -> make array number based on data; 1 -> grayscale (3 if RGB)

ValueError: cannot reshape array of size 41900 into shape (50,50,1)

Answer 1

You are iterating through the dataset twice with this line:

for category in CATEGORIES: ## CATEGORIES = ['dog', 'cat']

In this sense, you adding everything twice and are also mislabeling the data twice - the first time through it will mark everything as "dog", then the second time "cat".

Also check this line..it's not doing what you want as it does not modify the array inplace.

cv2.resize(cat_img_array, (IMG_SIZE, IMG_SIZE))

What if you just did something like this. You can also get rid of the if statement.

CATEGORIES = {'cat': 0, 'dog': 1}
for img in os.listdir(train_path):
    animal = img.split('.')[0]
    img_array = cv2.imread(os.path.join(train_path, img), cv2.IMREAD_GRAYSCALE)  # convert to grayscale bc RGB > gray; plus not essential
    resized = cv2.resize(img_array, (IMG_SIZE, IMG_SIZE))
    training_data.append([resized, CATEGORIES[animal]])