# Why are all weights of a neural net updated and not just the weights of the first layer

Why are all weights of a neural net updated and not only the weights of the first hidden layer?

The error-influence of the prediction by the weights of a neural net is calculated using the chain rule. However, the chain rule tells us how the first variable influences the second variable, and so on. Following that logic, we should only update the weights of the first hidden layer. My thought is, that if we backtrack the influence of the first variable but also change the values of the subsequent weights (of the subsequent hidden layer), there is no need to calculate the influence of the first weights in the first place. Where am I wrong?

Each weight $$w_i$$ (not just the ones in the first layer) affects the loss $$\mathcal{L}$$ according to the partial derivative of $$\mathcal{L}$$ with respect to $$w_i$$, i.e. $$\frac{\partial \mathcal{L}}{\partial w_i}$$. Intuitively, the partial derivative with respect to parameter tells you how the function is changing with respect to that parameter.
Why do you do this? In this case, the loss function is multi-variable function, so it depends on multiple variables. The gradient $$\nabla \mathcal{L} = \left[ \frac{\partial \mathcal{L}}{\partial w_1}, \dots, \frac{\partial \mathcal{L}}{\partial w_N} \right]$$ represents the direction (note that the gradient is a vector and vectors have direction) towards which your function is increasing or decreasing (depending on the sign of the gradient).