Moving_mean and Moving_variance in BatchNormalization layer of Keras

Question

I want to export a set of pre-trained weights from Tensorflow to Keras. The problem is that batch normalization layers in Tensorflow embed only Beta and Gamma as trainable weights, whereas in Keras, we have Moving_mean and Moving_variance as well. I am confused where to obtain these weights from.

Answer 1

Try tf.train.NewCheckpointReader. I've converted a CNN model from TF to Keras recently, and there is no problem exporting the moving mean/variance weights with it.

reader = tf.train.NewCheckpointReader(ckpt_file)
for key in reader.get_variable_to_shape_map():
    path = os.path.join(output_folder, get_filename(key))
    arr = reader.get_tensor(key)
    np.save(path, arr)
    print("tensor_name: ", key)

where get_filename() is just a function converting tensor names to proper filenames. (e.g., replacing slashes with underscores)

The full code may be helpful if you're interested in more details.