I understand that this is the method for conducting back-propogation:

  • With a three layer network (input layer, one hidden layer, one output layer); start with input $I_i$ as an exemplar input.
  • Calculate the output for this input (by multiplying by weights, checking against thresholds, and propogating through the network to the output); output = $O_k$
  • $y_k$ is the target output for $O_k$

To update the weights leading to this output in the final layer (i.e. $w_{jk}$; between the one hidden layer and the output layer):

  1. Calculate $\frac{\delta E}{\delta X_k} = y_k(1-y_k)(y_k-O_k)$
  2. Calculate $\frac{\delta E}{\delta w_{jk}} = \frac{\delta E}{\delta X_k}y_j$
  3. For all j,k; $w_{jk} := w_{jk} - C\frac{\delta E}{\delta w_{jk}}$

Then for the $w_{ij}$ weights (i.e. the weights between the input layer and the hidden layer):

  1. Calculate all the $\frac{\delta E}{\delta X_j} = y_j(1-y_j) \sum_k \frac{\delta E}{\delta X_k} w_{jk} $
  2. Calculate all the $\frac{\delta E}{\delta w_{ij}} = \frac{\delta E}{\delta x_j}I_i$
  3. For all i,j: $ w_{ij} = w_{ij} - C\frac{\delta E}{\delta w_{jk}}$

What I understand:

  1. I understand what step 3 and 6 is doing. You have the weight, a learning rate ($C$) and the error you've calculated to be associated with that weight, and you're updating the weight to reflect the error.

  2. For step 1,2,4 and 5, I understand the left hand side of each equation: 1 is measuring the change in total error for that input X, 2 is measuring total error for the weight $w_{jk}$, 5 is measuring the total error for the weight $w_{ij}$ and 4 is measuring the error for that X value.

What I don't understand is, in plain basic english, what the right hand side of equations 1,2,4 and 5 are doing. Can someone else with this?

  • $\begingroup$ I would suggest that, in this post, you focus on a specific equation, maybe the first one that you need to understand before understanding the others, so that people can focus on a single equation at a time. So, I would actually suggest that you split this post into multiple ones. And try to put a more specific question in the title. $\endgroup$
    – nbro
    Jan 4 at 22:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.