Search code examples
network-programmingneural-networkcelllstm

Where does the LSTM cell go?

I've read and seen a lot of videos about the LSTM cell and I understand that it uses various gates to manipulate the memory or cell state that it has:

this picture)

My question is Where does this LSTM cell go?

Does it:

  1. Replace each perceptron? I'd think not because it seems to receive a vector input.
  2. Get placed between hidden layers?
  3. Placed after the final output is generated and the cell state is passed to the input layer?

Or something else entirely?

I found this picture but I'm still confused. Any help would be appreciated.

this picture

Solution

  • Hello If I understood your question, you are asking about where are placed the LSTM cells in a neural network.

    Well, a neural network is composed from input,hidden and output layers. The LSTM layer may be placed in anyone of them.

    Now, each LSTM layer is composed of cells. These cells represents the perceptrons of the LSTM layer, they work differently from the traditional perceptrons.

    As you see in this picture, the perceptrons of the reccurent layer takes into consideration the output of the previous timestep and the new input. Based on how these two parameters are used to calculate the new output, we have many types of RNNs, like LSTM and GRU.

    enter image description here

    I hope this answers your question.