Silva and Almeida's algorithm improves on the existing backpropagation algorithm by introducing individual, adaptive learning-rates for each weight. The value for the new learning rate is computed as follows:
I read that the constants u
and d
are set to be u > 1
and d < 1
. Those constraints are rather broad, so are there any general guidelines for setting these values or do I have to figure it out by experimentation for my specific problem?
I have read that good "starting" values to fit most problems are to try u = 1.2 and d = 0.8 but i can't find the source right now.
Edit: I found it, PDF page 10-11
Also note the comments about how to improve upon the algorithm by introducing a momentum term, if you don't have that already...