I recently want to use batch normalization in keras to construct a neural network.As the original paper mentioned, the batch normalization behaves differently in testing and training time.I check the documentation:
keras.layers.BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001,
center=True, scale=True, beta_initializer='zeros', gamma_initializer='ones',
moving_mean_initializer='zeros', moving_variance_initializer='ones',
beta_regularizer=None, gamma_regularizer=None, beta_constraint=None,
gamma_constraint=None)
I didn't see any parameter to tell whether it is in training phase or testing phase,which is different form the implementation of tensorflow :
batch_norm(
inputs,
decay=0.999,
center=True,
scale=False,
epsilon=0.001,
activation_fn=None,
param_initializers=None,
param_regularizers=None,
updates_collections=tf.GraphKeys.UPDATE_OPS,
is_training=True,
reuse=None,
variables_collections=None,
outputs_collections=None,
trainable=True,
batch_weights=None,
fused=False,
data_format=DATA_FORMAT_NHWC,
zero_debias_moving_mean=False,
scope=None,
renorm=False,
renorm_clipping=None,
renorm_decay=0.99
)
I wonder how is batch normalization layer implemented in keras in testing phase? Thanks in advance.
You won't find anything about training x testing in the documentation.
Keras hides those in its source code, and the idea is:
fit and similars in a model, keras behaves like in training mode; evaluate, predict and others, it behaves like not in training. If you want to find these differences for the BatchNormalization layer, you will have to inspect its source code.
There is something about it here, I guess. But I won't pretend I understand it well: https://github.com/fchollet/keras/blob/master/keras/layers/normalization.py#L170