Search code examples
pythonmachine-learningkeraskeras-layerautoencoder

Extracting encoding/decoding models from Keras autoencoder using Sequential API

I am training an autoencoder constructed using the Sequential API in Keras. I'd like to create separate models that implement the encoding and decoding functions. I know from examples how to do this with the functional API, but I can't find an example of how it's done with the Sequential API. The following sample code is my starting point:

input_dim = 2904
encoding_dim = 4
hidden_dim = 128

# instantiate model
autoencoder = Sequential()

# 1st hidden layer    
autoencoder.add(Dense(hidden_dim, input_dim=input_dim, use_bias=False))
autoencoder.add(BatchNormalization())
autoencoder.add(Activation('elu'))
autoencoder.add(Dropout(0.5))

# encoding layer    
autoencoder.add(Dense(encoding_dim, use_bias=False))
autoencoder.add(BatchNormalization())
autoencoder.add(Activation('elu'))
# autoencoder.add(Dropout(0.5))

# 2nd hidden layer    
autoencoder.add(Dense(hidden_dim, use_bias=False))
autoencoder.add(BatchNormalization())
autoencoder.add(Activation('elu'))
autoencoder.add(Dropout(0.5))

# output layer
autoencoder.add(Dense(input_dim))

I realize I can select individual layers using autoencoder.layer[i], but I don't know how to associate a new model with a range of such layers. I naively tried the following:

encoder = Sequential()
for i in range(0,7):
    encoder.add(autoencoder.layers[i])

decoder = Sequential()
for i in range(7,12):
    decoder.add(autoencoder.layers[i])


print(encoder.summary())
print(decoder.summary())

which seemingly worked for the encoder part (a valid summary was shown), but the decoder part generated an error:

This model has not yet been built. Build the model first by calling build() or calling fit() with some data. Or specify input_shape or batch_input_shape in the first layer for automatic build.
Solution

  • Since the input shape for a middle layer (i.e. here I am referring to autoencoder.layers[7]) is not explicitly set, when you add it to another model as the first layer, that model would not be built automatically (i.e. building process involves constructing weight tensor for the layers in the model). Therefore, you need to call build method explicitly and set the input shape:

    decoder.build(input_shape=(None, encoding_dim))   # note that batch axis must be included

    As a side note, there is no need to call print on model.summary(), since it would print the result by itself.