How to use deconvolution with MNIST database

Question

I am a newbie in CNN and I am trying the code the Deconvolution (to generate feature maps) in MNIST database (because it's the simplest one to learn for a beginner). I want my model to generate feature maps at the end.The idea is to implement the paper Saliency Detection Via Dense Convolution Network to some extent.

Here is the complete code that I am trying to run:

import keras
from keras.datasets import mnist
import keras.backend as K
from keras.models import Model, Sequential
from keras.layers import Input, Dense, Flatten, Dropout, Activation, Reshape
from keras.layers.advanced_activations import LeakyReLU
from keras.layers.pooling import MaxPooling2D, GlobalAveragePooling2D
from keras.layers.normalization import BatchNormalization
from keras.layers.convolutional import Conv2D, Conv2DTranspose, UpSampling2D
from keras.initializers import RandomNormal

init = RandomNormal(mean = 0., stddev = 0.02)

def GeneratorDeconv(image_size = 28): 

    L = int(image_size)

    inputs = Input(shape = (100, ))
    x = Dense(512*int(L/16)**2)(inputs) #shape(512*(L/16)**2,)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Reshape((int(L/16), int(L/16), 512))(x) # shape(L/16, L/16, 512)
    x = Conv2DTranspose(256, (4, 4), strides = (2, 2),
                        kernel_initializer = init,
                        padding = 'same')(x) # shape(L/8, L/8, 256)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Conv2DTranspose(128, (4, 4), strides = (2, 2),
                        kernel_initializer = init,
                        padding = 'same')(x) # shape(L/4, L/4, 128)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Conv2DTranspose(64, (4, 4), strides = (2, 2),
                        kernel_initializer = init,
                        padding = 'same')(x) # shape(L/2, L/2, 64)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Conv2DTranspose(3, (4, 4), strides= (2, 2),
                        kernel_initializer = init,
                        padding = 'same')(x) # shape(L, L, 3)
    images = Activation('tanh')(x)

    model = Model(inputs = inputs, outputs = images)
    model.summary()
    return model

batch_size = 128
num_classes = 10
epochs = 1

# input image dimensions
img_rows, img_cols = 28, 28

# the data, split between train and test sets
(x_train, y_train), (x_test, y_test) = mnist.load_data()

if K.image_data_format() == 'channels_first':
    x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols)
    x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols)
    input_shape = (1, img_rows, img_cols)
else:
    x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)
    x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)
    input_shape = (img_rows, img_cols, 1)

x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255
print('x_train shape:', x_train.shape)
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')

# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)


model = GeneratorDeconv()

model.compile(loss=keras.losses.categorical_crossentropy,
              optimizer=keras.optimizers.Adadelta(),
              metrics=['accuracy'])

model.fit(x_train, y_train,
          batch_size=batch_size,
          epochs=epochs,
          verbose=1,
          validation_data=(x_test, y_test))

score = model.evaluate(x_test, y_test, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])

The function def GeneratorDeconv(image_size = 28): I picked from ProgramCreek Python

Now, I am confused that how can I embed it into my own custom model. Up to model.compile(...) the program runs okay. But at model.fit(...) , it gives error:

ValueError: Error when checking input: expected input_2 to have 2 dimensions, but got array with shape (60000, 28, 28, 1)

I don't know how to resolve the issues. Please help.

Answer 1

The input to your model is:

    inputs = Input(shape = (100, ))

This will take a vector in the shape of (samples, 100), so it expects a 2D input.

However:

print('x_train shape:', x_train.shape)
>>>x_train shape: (60000, 28, 28, 1)

You are inputting a 4D array, when you specified that your input took a 2D one. That is what is causing the error.

I made some edits to your architecture so the shapes match up and it actually trains:

def GeneratorDeconv(image_size = 28):

    L = int(image_size)

    inputs = Input(shape = (28, 28,1))
    x = Dense(512*int(L/16)**2)(inputs)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Conv2DTranspose(256, (4, 4), strides = (2, 2),
                    kernel_initializer = init,
                    padding = 'same')(x)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Conv2DTranspose(128, (4, 4), strides = (2, 2),
                    kernel_initializer = init,
                    padding = 'same')(x)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Conv2DTranspose(64, (4, 4), strides = (2, 2),
                    kernel_initializer = init,
                    padding = 'same')(x)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    x = Conv2DTranspose(3, (4, 4), strides= (2, 2),
                    kernel_initializer = init,
                    padding = 'same')(x)
    x = Flatten()(x)
    x = Dense(10)(x)
    images = Activation('tanh')(x)

    model = Model(inputs = inputs, outputs = images)
    model.summary()
    return model