Search code examples
keraslstmrecurrent-neural-network

How to properly set the input_shape of LSTM layers?


I have an input data with the following shape:(5395, 69, 1)

Should my input_shape be:

  • (69,1) or

  • (1,69) ?

With 69 neurons in the LSTM layer I get in the first input_shape 19'596 parameters to train, and with the second 38'364 parameters, aren't those the result of get as input 1 and 69 values, respectively? My question is should I have as input 1 because I have 1 feature or 69 because I have 69 timesteps, and why?


Solution

  • The input of LSTM layer has a shape of (num_timesteps, num_features), therefore:

    • If each input sample has 69 timesteps, where each timestep consists of 1 feature value, then the input shape would be (69, 1).

    • If each input sample is a single timestep of 69 feature values, then probably it does not make sense to use an RNN layer at all since basically the input is not a sequence. Instead, it's better to flatten the input sample (i.e. reshape (1, 69) to (69,)) and then use other connectivity architectures/layers (e.g. Dense).


    As a side note, I might be wrong, but I have a feeling that you are mixing the number of input timesteps and the number of units/neurons in LSTM layer (specifically, I am referring to this sentence of yours: "With 69 neurons in the LSTM layer..."). These two have nothing to do with each other and they should not necessarily be the same number. The number of units/neurons in a LSTM layer determines the representational capability of that layer and should be set accordingly based on experiments/experience. This answer explains this point a bit further if you are interested.