How to use the first layers of a pretrained model to extract features inside a Keras model (Functional API)

Question

I would like to use the first layers of a pre-trained model --say in Xception up and including the add_5 layer to extract features from an input. Then pass the output of the add_5 layer to a dense layer that will be trainable.

How can I implement this idea?

Answer 1

Generally you need to reuse layers from one model, to pass them as an input to the rest layers and to create a Model object with input and output of the combined model specified. For example alexnet.py from https://github.com/FHainzl/Visualizing_Understanding_CNN_Implementation.git.

They have

from keras.models import Model

from keras.layers.convolutional import Conv2D, MaxPooling2D, ZeroPadding2D

def alexnet_model():
    inputs = Input(shape=(3, 227, 227))
    conv_1 = Conv2D(96, 11, strides=4, activation='relu', name='conv_1')(inputs)
    …
    prediction = Activation("softmax", name="softmax")(dense_3)
    m = Model(input=inputs, output=prediction)
    return m

and then they take this returned model, the desired intermediate layer and make a model that returns this layer’s outputs:

def _sub_model(self):
    highest_layer_name = 'conv_{}'.format(self.highest_layer_num)
    highest_layer = self.base_model.get_layer(highest_layer_name)
    return Model(inputs=self.base_model.input,
                 outputs=highest_layer.output)

You will need similar thing,

highest_layer = self.base_model.get_layer('add_5')

then continue it like

my_dense = Dense(... name=’my_dense’)(highest_layer.output)
…

and finish with

return Model(inputs=self.base_model.input,
             outputs=my_prediction)

Since highest_layer is a layer (graph node), not a connection, returning result (graph arc), you’ll need to add .output to highest_layer.

Not sure how exactly to combine models if the upper one is also ready. Maybe something like

model_2_lowest_layer = model_2.get_layer(lowest_layer_name)
upper_part_model = Model(inputs= model_2_lowest_layer.input,
                         outputs=model_2.output)
upper_part = upper_part_model()(highest_layer.output)
return Model(inputs=self.base_model.input,
             outputs=upper_part)