Search code examples
tensorflowobject-detection

Tensorflow Object Detection: use Adam instead of RMSProp


I'm training a CNN with this [.config file][1]:

rms_prop_optimizer: {
    learning_rate: {
      exponential_decay_learning_rate {
        initial_learning_rate: 0.004
        decay_steps: 800720
        decay_factor: 0.95
      }
    }
   momentum_optimizer_value: 0.9
   decay: 0.9
   epsilon: 1.0
}   

}
As you can see there is a rms_prop as optimizer. What if I would like to use Adam? How am I supposed to edit this file?


Solution

  • if I'm right, you're trying to use the object_detection model with a pre-trained network offered by Tensorflow, am I right? Then, if you know a little of programming, you can take a look at models/research/object_detection/builders/optimizer_builder.py and see which are the optimizer that can be used and with which parameters. Instead if you just want a out-of-the-box solution, this is how I did:

    optimizer {
        # momentum_optimizer {
        adam_optimizer: {
          learning_rate: {
            manual_step_learning_rate {
              initial_learning_rate: .0002
              schedule {
                step: 4500
                learning_rate: .0001
              }
              schedule {
                step: 7000
                learning_rate: .00008
              }
              schedule {
                step: 10000
                learning_rate: .00004
              }
            }
          }
          # momentum_optimizer_value: 0.9
        }
        use_moving_average: false
      }
    

    In my (little) experience I noticed that using the same learning_experience as momentum_optimizer makes the learning too fast and/or brings to NaN Losses, so I usually decrease it of 10 times or more. I'm trying just now. :)