I would like to know the difference between these 2 Models. the one above has 4 Layers looking into the model summary and you can also define the unit numbers for dimensionality reduction. But what is with the 2nd Model it has 3 layers and you cant directly define the number of hidden units? Are both LSTM Autoencoders for dimensionality reduction and regression analysis ? Are there any good papers describing these two examples that I found from keras andhere. I did nowhere defined the variables, infact that I am not asking for a coding question directly. I hope this also a good place for this topic. 1. Model:
from keras.layers import *
from keras.models import Model
from keras.layers import Input, LSTM, Dense, RepeatVector
samples=1000
timesteps=300
features=input_dim=1
data_shape=np.reshape(data,(samples,timestep,input_dim)
inputs = Input(shape=(timestep, input_dim))
encoded = LSTM(units, return_sequences=False, name="encoder")(inputs)
decoded = RepeatVector(timestep)(encoded)
decoded = LSTM(input_dim, return_sequences=True, name='decoder')(decoded)
autoencoder = Model(inputs, decoded)
encoder = Model(inputs, encoded)
print (autoencoder.summary())
2. Model:
x = np.random.random((1000, 300, 1))
2.model:
m = Sequential()
m.add(LSTM(100, input_shape=(300,1)))
m.add(RepeatVector(300))
m.add(LSTM(100, return_sequences=True))
print (m.summary())
m.compile(loss='mse', optimizer='rmsprop', metrics=['mse', 'mape'])
history = m.fit(x, x, nb_epoch=2000, batch_size=100)
When I try to add to both of them a data with the shape e.g. (1000, 300, 1) the first one is accepting it the second not, I get the error expected lstm_4 to have shape (None, 300, 100) but got array with shape (1000, 300, 1). With the choosen input_dim 1 and units =100. what am I doing wrong ? This is what I want to be:
LSTM(100, input_shape=(300, 1))
with units=100 When I run the model, I get the following error: Error when checking target: expected lstm_2 to have shape (None, 300, 100) but got array with shape (1000, 300, 1)
Where is my mistake that the model does not accept my data shape and my units size?
The two models have no structural difference; they both consist of an encoder followed by a decoder implemented by LSTM layers. The difference is notational; the first model is defined on the functional API with the input being considered a layer, whereas the second is defined using the sequential API. As for the encoder-decoder (otherwise known as seq2seq) architecture, it was originally proposed here, and has since evolved greatly, with the most significant improvement being the attention layer.