I made an autoencoder for dimensionality reduction, I wanted to save it to be used in the reduction of the test dataset. Here is my code
dom_state = seed(123)
print('Rescaling Data')
y = minmax_scale(X, axis=0)
ncol = y.shape[1] #here ncol = 19
print('Encoding Dimensions')
encoding_dim = 3
input_dim = Input(shape = (ncol,))
with tf.Session(config=tf.ConfigProto(intra_op_parallelism_threads=24)) as sess:
K.set_session(sess)
print('Initiating Encoder Layer')
encoded1 = Dense(20, activation = 'relu')(input_dim)
encoded2 = Dense(10, activation = 'relu')(encoded1)
encoded3 = Dense(5, activation = 'relu')(encoded2)
encoded4 = Dense(encoding_dim, activation = 'relu')(encoded3)
print('Initiating Decoder Layer')
decoded1 = Dense(5, activation = 'relu')(encoded4)
decoded2 = Dense(10, activation = 'relu')(decoded1)
decoded3 = Dense(20, activation = 'relu')(decoded2)
decoded4 = Dense(ncol, activation = 'sigmoid')(decoded3)
print('Combine Encoder and Decoder layers')
autoencoder = Model(input = input_dim, output = decoded4)
print('Compiling Mode')
autoencoder.compile(optimizer = 'Nadam', loss ='mse')
autoencoder.fit(y, y, nb_epoch = 300, batch_size = 20, shuffle = True)
encoder = Model(input = input_dim, output = decoded4)
encoder.save('reduction_param.h5')
print('Initiating Dimension Reduction')
model = load_model('reduction_param.h5')
encoded_input = Input(shape = (encoding_dim, ))
encoded_out = model.predict(y)
However, even if I have restricted the dimensions, on the model.predict(y) part, I still get the full 19 columns instead of 3. Furthermore, I am also receiving the error:
UserWarning: No training configuration found in save file: the model was *not* compiled. Compile it manually.
warnings.warn('No training configuration found in save file:
which I understand, because, the encoder.save('reduction_param.h5')
is actually not compiled with an optimiser. Am I missing something?
EDIT:
I do not know if this is the correct way to address the issue, basically I train the MinMAXScaler() to the training dataset, saved the features as a pickle, then re-use it instead while maintaining the auto-encoder, as per code:
dom_state = seed(123)
print('Rescaling Data')
feature_space= MinMaxScaler()
feature_pkl = feature_space.fit(X)
filename = 'lc_feature_space.sav'
pickle.dump(feature_pkl, open(filename, 'wb'))
loaded_model = pickle.load(open(filename, 'rb'))
y = loaded_model.transform(X)
ncol = y.shape[1]
print(ncol)
print('Encoding Dimensions')
encoding_dim = 3
input_dim = Input(shape = (ncol,))
with tf.Session(config=tf.ConfigProto(intra_op_parallelism_threads=24)) as sess:
K.set_session(sess)
print('Initiating Encoder Layer')
encoded1 = Dense(20, activation = 'relu')(input_dim)
encoded2 = Dense(10, activation = 'relu')(encoded1)
encoded3 = Dense(5, activation = 'relu')(encoded2)
encoded4 = Dense(encoding_dim, activation = 'relu')(encoded3)
print('Initiating Decoder Layer')
decoded1 = Dense(5, activation = 'relu')(encoded4)
decoded2 = Dense(10, activation = 'relu')(decoded1)
decoded3 = Dense(20, activation = 'relu')(decoded2)
decoded4 = Dense(ncol, activation = 'sigmoid')(decoded3)
print('Combine Encoder and Deocoder layers')
autoencoder = Model(input = input_dim, output = decoded4)
print('Compiling Mode')
autoencoder.compile(optimizer = 'Nadam', loss ='mse')
autoencoder.fit(y, y, nb_epoch = 300, batch_size = 20, shuffle = True)
print('Initiating Dimension Reduction')
encoder = Model(input = input_dim, output = decoded4)
encoded_input = Input(shape = (encoding_dim, ))
encoded_out = encoder.predict(y)
result = encoded_out[0:2]
my argument here is to save the features of the training dataset at the MINMAXScaler() level, transform the test dataset based on these features, then just reduce using the auto-encoder. Still I do not know if this is correct.
I think the reason why you didn't see encoder
to work properly, i.e. reducing an input tensor's dimensionality, is because you defined and saved a wrong model. You should use
encoder = Model(input = input_dim, output = encoded4 )
whose output node is encoded4
instead of decoded4
.