Save features from hidden layers from an auto-encoder model using keras in runtime

Question

I am training an auto-encoder model and want to save the features of each image during runtime from encoder part and use it later for feature matching.

My model structure is-

train_data_dir = '/train'
test_data_dir = '/test'
nb_train_samples = 100
nb_validation_samples = 25
nb_epoch = 2
batch_size = 5
input_img = Input(shape=( img_width, img_height,3))

x = Conv2D(128, (3, 3), activation='relu', padding='same')(input_img)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(64, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(32, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(16, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
encoded = MaxPooling2D((2, 2), padding='same')(x)

x = Conv2D(8, (3, 3), activation='relu', padding='same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Conv2D(16, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(32, (3, 3), activation='relu',padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(64, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(128, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
decoded = Conv2D(3, (3, 3), activation='sigmoid', padding='same')(x)
autoencoder = Model(input_img, decoded)
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')

def fixed_generator(generator):
    for batch in generator:
        yield (batch, batch)

train_datagen = ImageDataGenerator(
        rescale=1./255,
        shear_range=0.2,
        zoom_range=0.2,
        horizontal_flip=True)
# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
        train_data_dir,
        target_size=(img_width, img_height),
        batch_size=batch_size,
        class_mode=None)
#print(type(train_generator))
test_generator = test_datagen.flow_from_directory(
        test_data_dir,
        target_size=(img_width, img_height),
        batch_size=batch_size,
        class_mode=None)

autoencoder.fit_generator(
        fixed_generator(train_generator),
        epochs=nb_epoch,
        samples_per_epoch=nb_train_samples,
        validation_data=fixed_generator(test_generator),
        validation_steps=nb_validation_samples
        )

How can i save the features from encoder part during model fit. Is there any way, please suggest

Answer 1

To save the features from a autoencoder model, first you need to load the model and extract the last encoder layer from the model.

Here is the code to extract the layers of an encoder and save the features for an image:

autoencoder= load_model('model_weights.h5')
encoder = Model(autoencoder.input, autoencoder.layers[-1].output)

# Read your input image for which you need to extract features

img = cv2.imread(img)
newimg = cv2.resize(img,(512,512))
encoded_imgs = encoder.predict(newimg[None])[0]

# Save your features as an numpy array

np.save('features.npy', encoded_imgs)

After saving your feature for an input image you need to find the euclidean distance for feature matching.

file = 'sample.npy'
file = np.load(file)
file1 = "features.npy"
file1 = np.load(file1)
distance = np.linalg.norm(file-file1)
print(distance)

This code will extract the features from an image and calculate the distance between two numpy arrays.