Search code examples
artificial-intelligencemachine-learningneural-networkbackpropagation

What are good values for u and d in Silva and Almeida's backpropagation algorithm?


Silva and Almeida's algorithm improves on the existing backpropagation algorithm by introducing individual, adaptive learning-rates for each weight. The value for the new learning rate is computed as follows:

Learning constant for the next step

I read that the constants u and d are set to be u > 1 and d < 1. Those constraints are rather broad, so are there any general guidelines for setting these values or do I have to figure it out by experimentation for my specific problem?


Solution

  • I have read that good "starting" values to fit most problems are to try u = 1.2 and d = 0.8 but i can't find the source right now.

    Edit: I found it, PDF page 10-11

    Also note the comments about how to improve upon the algorithm by introducing a momentum term, if you don't have that already...