How to initialize weight for convolution layers in Tensorflow Object Detection API?

Question

I followed this tutorial for implementing Tensorflow Object Detection API.

The preferred way is using pretrained models.

But for some cases, we need to train from scratch.

For that we just need to comment out two lines in the configuration file as

#fine_tune_checkpoint: "object_detection/data/mobilenet_v1_1.0_224/mobilenet_v1_1.0_224.ckpt"
#from_detection_checkpoint: true 

If I want to initialize weight with Xavier weight initialization, how can I do that?

Answer 1

As you can see in the configuration protobuf definition, there are 3 initializers you can use:

• TruncatedNormalInitializer truncated_normal_initializer
• VarianceScalingInitializer variance_scaling_initializer
• RandomNormalInitializer random_normal_initializer

The VarianceScalingInitializer is what you are looking for. It is general initializer which you can basically turn into Xavier initializer by setting factor=1.0, mode='FAN_AVG', as stated in the documentation.

So, by setting the initializers as

initializer {
    variance_scaling_initializer {
        factor: 1.0
        uniform: true
        mode: FAN_AVG
    }
}

in your configuration, you obtain Xavier initializer.

But also, even if you need to train on new data, consider using pretrained network as initialization instead of random initialization. For more details, see this article.