Finding the best learning rate in tensorflow object detection

Question

I want to search for the best learning rate using tensorflow object detection api. But in the config file I'm not able to find anything for it. I can add schedule but it can't search for the best learning rate.

learning_rate: {
            manual_step_learning_rate {
              initial_learning_rate: 0.003
              schedule {
                step: 6000
                learning_rate: .0003
              }
              schedule {
                step: 12000
                learning_rate: .00003
              }

Is there any trick or way to search for best learning rate.

Answer 1

If you refer to the Learning Rate Finder (as described by Smith for example here: https://arxiv.org/abs/1803.09820), it seems like you can emulate it by using:

learning_rate: {
    exponential_decay_learning_rate {
          initial_learning_rate: 0.004
          decay_steps: 10000
          decay_factor: 1.3
    }
}

with a decay_factor above 1.

You will still have to look at the loss and choose the best learning rate yourself though.