Search code examples
pythontensorflowkeras

How to re-initialize weights of pre-trained model with Glorot on Keras?

I'm trying to re-initialice layers weights using Glorot Uniform with Keras from Tensorflow. The closest approach is this:

import numpy as np
import tensorflow as tf
for layer in base_model.layers:
  layer_new_weights = []
  for layer_weights in layer.get_weights():
    initializer = tf.compat.v1.keras.initializers.glorot_normal
    weights = initializer(np.shape(layer_weights))
    layer_new_weights .append(weights)
  layer.set_wegiths(layer_new_weights)

Any idea how to really set weights initializing from Glorot Uniform each layer of pretrained model as ResNet50?

Thanks!

Solution

  • A general solution with less code is the following:

    init = tf.keras.initializers.GlorotUniform()
for w in model.trainable_variables:
    w.assign(init(w.shape))