Search code examples
pythontensorflowinputrecurrent-neural-networktraining-data

Will model.predict update a GRU layer states correctly?

This is my first question, suggestions are appreciated!

Background

I built a model similar to this one. my model

I am using model.predict(inputs) to make predictions and I built my custom functions and process to make it learn. It seems to work and when it predicts all seems good.

Some days ago, I watched this video: https://youtu.be/8HyCNIVRbSU?t=284 (It starts in the part that generated my question) And I copied the image that caught my attention: LSTM It says that each input is inserted sequentially on each cell, and each cell will provide more information to the next in that order.

I began to wonder if my implementation is correct and I will give you all the details that might be useful.

Details

  1. I am using stateful=True for both GRU layers.
  2. The input at model.predict(inputs) is defined like inputs = [np.array(arr).reshape(1, 1, 6), np.array(arr2).reshape(1, 1, 4)].
  3. I have a CSV with the values of the first part of the input, but for the other values, I create them on the fly. That is why I used model.predict. Because of this, if there is another way to be able to send both the static and "dynamic data", I would be happy to do it.
  4. The only way that I am interacting with the model after loading it is by asking to predict the values, and by updating its weights (and resetting states when I input a new sequence).

Now, with all this information, the question would be:

  • Is model. predict updating all the cell states correctly and using the GRU layer to its full capacity?
  • Or is it just sending the data to the first cell always?
Solution

  • In a course of forward pass (either predict() or fit()) GRU (and other RNNs) takes the first item from your sequence, recalculates its state, takes the second item, recalculates its state and so on.