ValueError: Input 0 of layer lstm_1 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: (None, 64)

I've been having a hard time comprehending what this error message is about. I've looked into many posts like

ValueError: Input 0 of layer sequential is incompatible with the layer: : expected min_ndim=4, found ndim=3

ValueError: Input 0 is incompatible with layer lstm_13: expected ndim=3, found ndim=4

Input 0 of layer sequential is incompatible with the layer expected ndim=3, found ndim=2. Full shape received: [None, 1]

expected ndim=3, found ndim=2

but none of them seem to resolve my problem.

I have

batch_train_dataset = tf.data.Dataset.from_tensor_slices((train_features, train_labels)).shuffle(512).batch(batch_size)

for i,x in enumerate(batch_train_dataset):
  print("x[0].ndim: ", x[0].ndim)
  print("x[0].shape: ", x[0].shape)
  print("x[1].shape: ", x[1].shape)
  if i==0:
    break
##########OUTPUT###########
x[0].ndim:  3
x[0].shape:  (64, 32, 1000)
x[1].shape:  (64,)

and my individual data has a shape(64,32,1000) where 64 is batch_size, 32 is timesteps and 1000 is a number of features.

This is my model.

num_classes = len(index_to_label)

lstm_model = tf.keras.Sequential([
    tf.keras.layers.Masking(mask_value=0.0), # DO NOT REMOVE THIS LAYER

    # TODO: Define a recurrent neural network to recognize one of `num_classes` actions from the given video
    ### START CODE HERE ###
    tf.keras.layers.LSTM(64, input_shape=(batch_size,32,1000)),
    tf.keras.layers.LSTM(32),
    tf.keras.layers.Dense(32, activation='relu'),
    tf.keras.layers.Dense(32, activation='relu'),
    tf.keras.layers.Dense(num_classes, activation='softmax')
    ### END CODE HERE ###
])

I think I had my input_shape set right and I have no idea what to fix here based on those questions that I listed above. Whenever I try to fit the model, it still prints an error

ValueError: Input 0 of layer lstm_1 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: (None, 64)

Can anyone help me with this please.

====================EDIT==================

Thanks to some comments, I've changed my input_shaped to (32,1000) but it still prints out the exact same error. I'm still wondering what the cause might be so any other ideas would be appreciated.

Solution

In the stacked LSTM layer, LSTM layers will not return sequences, i.e., they will return 2D output. Which means that the second LSTM layer will not have the 3D input it needs. To address this, you need to set the return_sequences=True:

tf.keras.layers.LSTM(64, input_shape=(32,1000),return_sequences=True),
tf.keras.layers.LSTM(32),