ValueError: Input 0 of layer lstm_28 is incompatible with the layer: expected ndim=3, found ndim=4. Full shape received: [None, 50, 21, 8]

Question

Input dimensions:

train_ohe :(95036, 50, 21)
y_train :(95036,)
val_ohe :(9911, 50, 21)
y_val :(9911,)

Model :

x_input = Input(shape=(50,))
emb = Embedding(21, 8, input_length=50)(x_input)
bi_rnn = LSTM(50, input_shape=(50,20,50), return_sequences=False)(emb)
x = Dropout(0.3)(bi_rnn)
x_output = Dense(1, activation='relu')(x)
model1 = Model(inputs=x_input, outputs=x_output)
model1.compile(optimizer='adam', loss='mse', metrics=['mse'])

history1 = model1.fit(
    train_ohe, y_train,
    epochs=1, batch_size=256,validation_data=(val_ohe, y_val),callbacks=[es]
    )

Note

Vocab size=20, input_length=50 

I tried changing return_sequences for True/False and also tried dimensions of LSTM layer from (50, 20)/(50,20,50) but still got the same error.

Answer 1

Looks like you are feeding one-hot vectors to Embedding layer. You should feed sequences of integers. Shape should be (95036, 50).

If you really need to feed one-hot vectors - you have to convert one-hot to integer to feed them into Embedding layer. Try something like that:

x_input = Input(shape=(50, 21))
x = tf.argmax(x_input, axis=-1)
emb = Embedding(21, 8, input_length=50)(x)