I am trying to implement a denoising autoencoder with an LSTM layer in between. The architecture goes following.
FC layer -> FC layer -> LSTM cell -> FC layer -> FC layer.
I am unable to understand how my input dimension should be to implement this architecture?
I tried the following code
batch_size = 1
model = Sequential()
model.add(Dense(5, input_shape=(1,)))
model.add(Dense(10))
model.add(LSTM(32))
model.add(Dropout(0.3))
model.add(Dense(5))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(trainX, trainY, nb_epoch=100, batch_size=batch_size, verbose=2)
My trainX is [650,20,1] vector. It is a time series data in with only one feature.
I am getting following error
ValueError Traceback (most recent call last)
<ipython-input-20-1248a33f6518> in <module>()
3 model.add(Dense(5, input_shape=(1,)))
4 model.add(Dense(10))
----> 5 model.add(LSTM(32))
6 model.add(Dropout(0.3))
7 model.add(Dense(5))
/usr/local/lib/python2.7/dist-packages/keras/models.pyc in add(self, layer)
330 output_shapes=[self.outputs[0]._keras_shape])
331 else:
--> 332 output_tensor = layer(self.outputs[0])
333 if isinstance(output_tensor, list):
334 raise TypeError('All layers in a Sequential model '
/usr/local/lib/python2.7/dist-packages/keras/engine/topology.pyc in __call__(self, x, mask)
527 # Raise exceptions in case the input is not compatible
528 # with the input_spec specified in the layer constructor.
--> 529 self.assert_input_compatibility(x)
530
531 # Collect input shapes to build layer.
/usr/local/lib/python2.7/dist-packages/keras/engine/topology.pyc in assert_input_compatibility(self, input)
467 self.name + ': expected ndim=' +
468 str(spec.ndim) + ', found ndim=' +
--> 469 str(K.ndim(x)))
470 if spec.dtype is not None:
471 if K.dtype(x) != spec.dtype:
ValueError: Input 0 is incompatible with layer lstm_10: expected ndim=3, found ndim=2
The dense layer can take sequences as input and it will apply the same dense layer on every vector (last dimension). Example :
You have a 2D tensor input that represents a sequence (timesteps, dim_features), if you apply a dense layer to it with new_dim outputs, the tensor that you will have after the layer will be a new sequence (timesteps, new_dim)
If you have a 3D tensor (n_lines, n_words, embedding_dim) that can be a document, with n_lines lines, n_words words per lines and embedding_dim dimensions for each word, applying a dense layer to it with new_dim outputs will get you a new doc tensor (3D) with shape (n_lines, n_words, new_dim)
You can see here the dimensions input and output that you can feed and get with the Dense() layer.