Search code examples
kerasdeep-learninglstmmulticlass-classification

Dimension issues for LSTM sequence model on Keras

I'd like to train a simple LSTM model for sequence data with 128 time steps with 6 features for 118 multi-classes.

The dimensions of the dataset are shown below:

X_train, X_test shape: (batch, timesteps, num_features) = (batch, 128, 6)
y_train, y_test shape: (batch, 118)

where labels are represented by one-hot encoding with 118 classes.

model = keras.Sequential([
    tf.keras.layers.LSTM(units = 32, kernel_initializer = 
    tf.initializers.zeros()),
    tf.keras.layers.Dense(units = 6)
])

model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(X_train, y_train, validation_data = (X_test, y_test), epochs=20, verbose=0)

There is an error below after executing the code above:

ValueError: Shapes (None, 118) and (None, 6) are incompatible

How to fix the dimension issue?

Solution

  • The units parameter in tf.keras.layers.Dense() is the dimensionality of the output space.

    Since you have used 6 units in the last dense layer, the model will process the input and return a tensor of shape (None, 6) and compare it with the labels of shape (None, 118), which would be incompatible.

    Try changing the number of units to 118 in the last dense layer to get a compatible model.