Search code examples
machine-learningkeraslstm

How keras.LSTM converts 3D input to 2D output?


From keras's LSTM documentation the input should be A 3D tensor with shape (batch, timesteps, feature)

The output will be (batch, units) where units is number features we want from LSTM unit.

As my knowledge single cell of lstm takes hidden state, cell state and single number as input for timestamp t and passes it's output to next cell in form of c(t+1) and h(t+1). But from the documentation code it is generating output in 2D form?

inputs = np.random.random((32, 10, 8))
lstm = keras.layers.LSTM(4)
output = lstm(inputs)
output.shape
(32, 4)

Question 1: How vector representation is passed to LSTM? (At each timestamp it is passing 8 features. If there are 8 lstm units running in parallel then output should also be of size 8)

Question 2: How the final output is of size 4. (If we ignore batch size)


Solution

  • Just like me, I found many people on Stackoverflow who are searching for the same answer on keras.LSTM and the answer to all those questions made me confused even more.

    The problem is, that every article and video talks about basic LSTM which takes one input per time step but in the real world we we have a vector representation.

    The complete flow of LSTM from input to output can be shown in the image below (which was my actual question)

    enter image description here

    Where in this case, the feature vector has a size of 3 and LSTM units have a size of 2 which makes input as (batch_size, time_steps, 3) and LSTM unit as LSTM(units=2). An explanation of the image can be found here to clear your further doubts.