I am creating my own recurrent layer and wish to allow dropout, therefore I want to be able define the dropout masks in a way similar to the way LSTM does it.
I note they define the mask in a method.
When and where is the keras.layers.recurrent.LSTMCell method _generate_dropout_mask is called in order to create the member variable dropout_mask?
I've searched the github repository for _generate_dropout_mask in an attempt to see if it called somewhere, and cannot find its mention anywhere except in the keras.layers.recurrent module.
Is there a part of training that perhaps looks for all callable layer attributes with a single underscore prepended, and runs them before keras.layer.build()? Since dropout is only used for training, and since the masks must be available before building I suspect this, as this might allow layers to define prebuild methods using undrescore notation.
The method is a member of the class SimpleRNNCell(Layer), L659.
The cell is a member of the SimpleRNN(RNN) class, defined at the init method, and passed to the super class RNN. Follow the lines: L919, L933, L339, L357.
The method is called inside the call method of a layer, L942.
The call method is called literally when you call a layer building your functional API Model. Not sure when the Sequential() model will call build. But it's certainly before, or at least immediately before, compiling. It's a phase that happens only once and links all the tensors together from input to output, creating the actual internal graph.
You can use dir(instance) to find all methods.
And you can use getattr(instance,methodName)(...) to call the method.
But build() is called before training. It's called when you call the layer for the first time when creating the graph with a functional API Model.