Search code examples
caffeconv-neural-networkgradient-descent

Should the learning rate be changed when the input/output has a different range?

I have an existing Caffe CNN which gets a 224*224 image as input, each pixel having a single value in the range [0,100]. The output has size 56*56, each being one of 313 classes.

Now I want to change the input/output type and train a model on this new network. The input values are in the range [0,1], and the output consists of 324 classes.

Should I change the base learning rate because of the changed input data range?

Solution

  • No, your learning rate has got nothing to do with your data. It is simply a factor by which the loss gradients are multiplied before updating the weights. You may consider it as a percentage, not absolute.