Data augmentation in Keras model

Question

I am trying to add data augmentation as a layer to a model but I am getting the following error.

TypeError: The added layer must be an instance of class Layer. Found: <tensorflow.python.keras.preprocessing.image.ImageDataGenerator object at 0x7f8c2dea0710>

data_augmentation = tf.keras.preprocessing.image.ImageDataGenerator(
   rotation_range=30, horizontal_flip=True)

model = Sequential()
model.add(data_augmentation)
model.add(Dense(1028,input_shape=(final_features.shape[1],)))
model.add(Dropout(0.7,input_shape=(final_features.shape[1],)))
model.add(Dense(n_classes, activation= 'softmax', kernel_regularizer='l2'))
model.compile(optimizer=adam,
              loss='categorical_crossentropy',
              metrics=['accuracy'])

history = model.fit(final_features, y,
            batch_size=batch_size,
            epochs=epochs,
            validation_split=0.2,
            callbacks=[lrr,EarlyStop]) 

I have also tried this way:

    data_augmentation = Sequential(
      [
        preprocessing.RandomFlip("horizontal"),
        preprocessing.RandomRotation(0.1),
        preprocessing.RandomZoom(0.1),
      ]
    )
    model = Sequential()
    model.add(data_augmentation)
    model.add(Dense(1028,input_shape=(final_features.shape[1],)))
    model.add(Dropout(0.7,input_shape=(final_features.shape[1],)))
    model.add(Dense(n_classes, activation= 'softmax', kernel_regularizer='l2'))
    model.compile(optimizer=adam,
                  loss='categorical_crossentropy',
                  metrics=['accuracy'])
    history = model.fit(final_features, y,
                batch_size=batch_size,
                epochs=epochs,
                validation_split=0.2,
                callbacks=[lrr,EarlyStop])

It gives an error:

    ValueError: Input 0 of layer sequential_7 is incompatible with the layer: expected ndim=4, found ndim=2. Full shape received: [128, 14272]

Could you please advice how I can use augmentation in Keras?

Answer 1

In your first case, you are using ImageDataGenerator as a layer, which is not: as the name says, it is just a generator which applies random transformations to images (image augmentation) before feeding the network. So, the images are augmented in CPU and then feed to the neural network which can run in GPU if you have one.

Generators are usually used also to avoid loading huge datasets into memory since they allow to load only the batches being used soon.

In the second case, you are using image augmentation as layers of your model properly. The difference here is that the augmentation is run as part of your model, so if you have a GPU available for instance, those operations will run in GPU.

The problem with your second case is in the model itself (in fact the model is also wrong in the first approach, you only get an error there with the bad usage of ImageDataGenerator before your execution arrives to the model).

Note that you are using images as inputs, so, the input should be of shape (height, width, channels), but then you are starting your model with a dense layer, which expects a single array of shape (n_features,).

If your model needs to start with a Dense layer (strange, but may be ok in some case) then you need first to use Flatten layer to convert images of shape (h,w,c) into vectors of shape (h*w*c,). This change will solve your second approach for sure.

That said, you don't need to specify the input shape on every single layer: doing it in your first layer should be enough.

Last, but not least: are you sure this model is being feed with images? According to your fit call, it looks like you are using previously extracted features that may be vectors (this make sense with your current model architecture but makes no sense with the usage of image augmentation).

Please, provide more details with respect to your data to clarify this point.