Input 0 is incompatible with layer lstm_12: expected ndim=3, found ndim=2

Question

I am new to ML and trying to make an RNN LSTM model.

I want to optimize the hyper-parameter using GridSearchCV. What I want to optimize is the number of layers and nodes for each number of layer selection.

Here is the code to generate the model:

from keras.wrappers.scikit_learn import KerasClassifier
from sklearn.model_selection import GridSearchCV
def create_model(layers,activation):
    model = Sequential()
    for i,node in enumerate(layers):
        if i == 0:
            model.add(LSTM(units=node, input_shape=(x_train.shape[1],1)))
            model.add(Activation(activation))
            model.add(Dropout(0.2))            
        else:
            model.add(LSTM(units=node, input_shape=(x_train.shape[1],1)))
            model.add(Activation(activation))
            model.add(Dropout(0.2)) 
    model.add(Dense(units=1)) 
    model.compile(optimizer='adam',loss='mean_squared_error',metrics=['accuracy'])
    return model

and here is the variables

layers=[[40,40],[30,30],[30,30,30],[30,30,30,30],[30,30,30,30,30]]
activations =['sigmoid','relu']
batch_size = [32,50]
epochs = [50]

then I wrap it up using gridsearchcv

param_grid = dict(layers=layers,activation=activations,batch_size=batch_size,epochs=epochs)
grid = GridSearchCV(estimator=model,param_grid=param_grid)

When I do it

grid_result = grid.fit(x_train,y_train,verbose=3)

I got this error

ValueError: Input 0 is incompatible with layer lstm_14: expected ndim=3, found ndim=2

I dont know what happens. My x_train shape is (13871, 60, 1) and y_train shape is (13871,). Thank you beforehand and your help will be very much appreciated!

Thanks!

Phil

Answer 1

The error message actually explains this well. LSTM requires a time series input of shape (batch_size, timesteps, features). You seem to have this correct for your first input lstm layer. However, the output of LSTM is not a sequence. Consequent LSTM layers will not receive appropriate input.

You can make the LSTM output also as a sequence by setting the parameter

return_sequences=True

Note that you may have to set return sequence to false in the final layer before dense or perform flatten operation.

Does that help?

PS: your if... else, condition are exactly the same. Is that something you plan to change later?