Shape incompatible error while using ImageDataGenerator for transfer learning

Question

I want to create a classification model. For this purpose I have collected some images from 3 different classes. First, I have implemented Xception model ( freezed all layers except the last one). However, it overfitted. Then, I have decided to use data augmentation strategy. This is the first time I have used Keras module for this purpose. I belive that I have correctly used it. But getting error ValueError: Shapes (None, None) and (None, None, None, 3) are incompatible. I have tried what I found from the web, but did not works. Can anyone point the what I am doing wrong? Here is the code.

from tensorflow import keras
from matplotlib import pyplot as plt
from keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.preprocessing import image
from tensorflow.keras.applications.imagenet_utils import preprocess_input
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Flatten, Activation
from tensorflow.keras.layers import Conv2D, MaxPooling2D
from tensorflow.keras.models import Model

# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(
        rescale=1./255,
        shear_range=0.2,
        zoom_range=0.2,
        horizontal_flip=True)

# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1./255)

# this is a generator that will read pictures found in
# subfolers of 'data/train', and indefinitely generate
# batches of augmented image data
train_generator = train_datagen.flow_from_directory(
        'data2/train',  # this is the target directory
        target_size=(299, 299),  # all images will be resized to 299x299 for the Xception
        batch_size=32,
        class_mode="categorical")  

# this is a similar generator, for validation data
validation_generator = test_datagen.flow_from_directory(
        'data2/validation',
        target_size=(299, 299),
        batch_size=32,
        class_mode="categorical")


Xception = keras.applications.Xception(weights='imagenet', include_top=False)
num_classes=3
inp = Xception.input
new_classification_layer = Dense(num_classes, activation='softmax')
out = new_classification_layer(Xception.layers[-2].output)
model_Xception = Model(inp, out)
model_Xception.summary()

for l, layer in enumerate(model_Xception.layers[:-1]):
    layer.trainable = False

for l, layer in enumerate(model_Xception.layers[-1:]):
    layer.trainable = True
    

opt=keras.optimizers.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-07)

model_Xception.compile(loss='categorical_crossentropy',
              optimizer=opt,
              metrics=['accuracy'])

model_Xception.summary()

model_Xception.fit_generator(
        train_generator,
        epochs=5,
        validation_data=validation_generator)
model_Xception.save_weights('first_try.h5') 

Answer 1

That's because you are feeding a convolution's output to a Dense layer.

You need to add one of Flatten, GlobalMaxPooling2D or GlobalAveragePooling2D in order to transform your output to (batch_size, input_size). You can change these lines:

inp = Xception.input
out_xception = Xception.layers[-2].output
flatten = tf.keras.layers.Flatten()(out_xception)
new_classification_layer = tf.keras.layers.Dense(num_classes, activation='softmax')
out = new_classification_layer(flatten)
model_Xception = tf.keras.Model(inp, out)
model_Xception.summary()

Second thing is since you did not specify input_shape while defining the Xception model, Flatten will throw an error. Simply change it to:

Xception = tf.keras.applications.Xception(weights='imagenet', include_top=False,
                                          input_shape = (299,299,3))