4
$\begingroup$

Why are the weights of a neural net updated only considering the old values of the later layer, not the already updated values?

I use this example to explain my problem. When applying the backpropagation chain rule, the weights of the previous layer ($w_1, w_2, w_3, w_4$) are updated making use of the chain rule:

$$\frac{\partial E_{total}}{\partial w_1} = \frac{\partial E_{total}}{\partial out_{h1}} * \frac{\partial out_{h1}}{\partial net_{h1}}*\frac{\partial net_{h1}}{\partial w_1}$$

He then says:

$$\frac{\partial net_{o1}}{\partial out_{h1}}=w_5$$

Although he has already calculated the updated value for $w_5$, he uses the old value of $w_5$ to update $w_1$? Because the updated value of $w_1$ will have an impact on the outcome together with the updated value of $w_5$?

$\endgroup$
1
  • $\begingroup$ Your idea is basically to use some intuitive kind of temporal difference, ie, use the difference between latest obtained value (by whatever method) and current value to update the attended independent variable's value, which is not used in backprop but reinforcement learning. $\endgroup$
    – cinch
    Nov 26, 2022 at 6:19

2 Answers 2

2
$\begingroup$

The basic idea of gradient descent is:

  • Calculate the gradient of some score with respect to parameters that you can control

  • Take a step in the direction of that gradient that improves the score (subract a multiple of gradient - for gradient descent - if you want to minimise some cost function)

The backpropagation using chain rule in neural network layers is part of the first part, calculating the gradient.

If you interleaved these steps during a single calculation, by taking an update step before propagating the gradient over all layers, you would not propagate the true gradient, but some interim value. There is no guarantee this value would reflect the true gradient of the affected layer when compared to training data and loss functions.

There are some manipulations of the gradient that are acceptable and improve convergence. For instance momentum, and various forms of weighting based on previous gradient calculations. However, I am not aware of any successful attempt to perform updates with partially-calculated gradient then attempt to continue the interrupted gradient calculations over the mixed updated and not-yet-updated parameters.

It is also possible to calculate gradients and then only update some of the possible parameters. In some cases this is useful, for instance it is a good choice when performing transfer learning, by only updating a few layers close to the output. This constrains the network to keep a lot of its trained parameters as-is, which reduces the chances of over-fitting to the smaller dataset that the network is learning from when transferring.

$\endgroup$
0
$\begingroup$

This is an excellent question, and the literature is generally silent on whether it's an improvement. I'm personally rather curious about this result!

There are some practical reasons it's not generally done:

Framework structure

Most ML frameworks separate the calculation of the gradient from the update, because users often want to mess with the gradient before updating the weights (e.g. Adam, momentum, etc). The way that most frameworks are implemented makes it difficult to do the optimizer calculation inline with the back-propagation.

Batch sharding

Most large networks have the batch calculation sharded over multiple chips. This means that when actually doing the gradient update, the partial sums need to be sent off-chip to be summed and added to the weights.

This normally isn't too bad because the transmission of the partials sums for one weight run in the parallel with the calculation of the gradients for the next weight.

But if we update the weights before back-propagating the gradient, then we need to wait for all the partial sums to be received and totaled, then the resulting update being transmitted back to all the chips before the next step in back-propagation can occur. This puts the network latency on the critical path, which makes everything very slow... So it's almost certain that it's faster to do the dumb thing with more steps, than the smarter thing with fewer steps. (The improvement in the gradient would have to save a huge fraction of steps to be worth it).

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .