# What is the derivative of a specific output with respect to a specific weight?

If I have a neural network, and say the 6th output node of the neural network is:

$$x_6 = w_{16}y_1 + w_{26}y_2 + w_{36}y_3$$

What does that make the derivative of:

$$\frac{\partial x_6}{\partial w_{26}}$$

I guess that it's how is $$x_6$$ changing with respect to $$w_{26}$$, so, therefore, is it equal to $$y_2$$ (since the output, $$y_2$$, will change depending on the weight added to the input)?

Formally speaking $$x_6$$ is a function of $$w_{16},\ w_{26}$$ and $$w_{36}$$, that is $$x_6 =f(w_{16}, w_{26}, w_{36})=w_{16}y_1 + w_{26}y_2 + w_{36}y_3.$$ The derivative w.r.t. $$w_{26}$$ is $$\frac{\partial x_6}{\partial w_{26}}= \frac{\partial w_{16}y_1}{\partial w_{26}} +\frac{\partial w_{26}y_2}{\partial w_{26}} +\frac{\partial w_{36}y_3}{\partial w_{26}} = 0 +y_2 \frac{\partial w_{26}}{\partial w_{26}} + 0= y_2.$$ The first equality is obtained using the fact that the partial derivative is linear (so the derivative of the sum is the sum of the derivatives); the second equality comes from again from the linearity and from the fact that $$w_{16}y_1$$ and $$w_{36}y_3$$ are constants with respect to $$w_{26}$$, so their partial derivative w.r.t. this variable is $$0$$.
Let's say $$x_6$$ is the output of the sixth node after you apply an activation function, that is $$x_6 =\sigma(f(w_{16}, w_{26}, w_{36}))=\sigma(w_{16}y_1 + w_{26}y_2 + w_{36}y_3).$$ You can compute the partial derivative applying the properties illustrated above, with the additional help of the chain rule $$\frac{\partial x_6}{\partial w_{26}}=\frac{\partial \sigma(w_{16}y_1 + w_{26}y_2 + w_{36}y_3)}{\partial w_{26}}=\sigma'\frac{\partial w_{16}y_1}{\partial w_{26}} +\sigma'\frac{\partial w_{26}y_2}{\partial w_{26}} +\sigma'\frac{\partial w_{36}y_3}{\partial w_{26}}=y_2\sigma'$$ $$\sigma'$$ denotes the derivative of sigma with respect to its argument.