How to update the error on a hidden node using back-propogation, given the error on the output nodes and weights

Question

I'm learning NN, and I want to manually code back-propogation to understand how it works, but I'm struggling with the algorithm. I'm trying to solve question 30 of this paper (so that I have an example of how it works to work from).

The short version of the question is if someone could show me how to do this to find the error for H2, I would appreciate it (the answer should be A; -0.0660).

The long version of the question is, is my thinking right (to find the error using back-propogation at H2):

1. The error (from question 29) for I1, I2 and I3 are 0.1479, -0.0929 and 0.1054, respectively.

2. The network architecture is:

3. The weights are:

So I thought what I had to do:

4. Find the total amount of weights that led to each output error (I took the absolute values to get the total sum of errors, is this right?):

   E1 = 1.0 + 3.5 => 4.5

   E2 = 0.5 + 1.2 => 1.7

   E3 = 0.3 + 0.6 => 0.9

5. So then I worked out the proportion of each weight that came from my node of interest (y2):

   E1 = 3.5/4.5 = 0.777

   E2 = 1.2/1.7 = 0.71

   E3 = 0.6/0.7 = 0.86

6. And then I worked out the proportion of error that came from that proportion of weight:

   E1 => (0.14/100)*14 = 0.01078

   E2 => (-0.09/100)*71 = -0.0639

   E3 => (0.1054/100)*86 = 0.090644

If someone could show me where I'm going wrong (because as mentioned above, I know what the right answer should be), I'd appreciate it. Also, as mentioned above, I've added a link to the original question 30 on the original exam, if it helps (it's from 17 years ago, not an exam I'm doing, just trying to understand it). I know that I can just use TensorFlow/Keras to implement this automatically, but I'm trying to really get into how it all works.

Answer 1

In the question you mentioned, you are given the error function:

You need to calculate its value for j = 2. You have the values for all delta_k and w_ij.

You are also given the derivative of activation function, f'(Hj):

Finally, you are given the activation of hidden node 2, which is f(H2). All you need to do is to place all the values you have into the equations:

f'(H2) = 0.74 * (1 - 0.74) = 0.1924

delta_2 = 0.1924 * ((0.1479 * -3.5) + (-0.0929 * -1.2) + (0.1054 * 0.6)) = -0.06598