# What is the proof behind the gradient of a curve being proportional to the distance between the two co-ordinates in the x-axis?

In the [delta rule][1] the equation to adjust the weight with respect to error is

$$w_{(n+1)}=w_{(n)}-\alpha \times \frac{\partial E}{\partial w}$$

*where $$\alpha$$ is the learning rate and $$E$$ is the error.

The graph for $$E$$ vs $$w$$ would look like the one below with $$E$$ in the $$y$$ axis and $$W$$ in the $$x$$-axis

In other words, we can write

$$\alpha \times \frac{\partial E}{\partial w}=w_{(n)}-w_{(n+1)}$$

I want to know, what is the proof behind the gradient of a curve being equal/proportional to the distance between the two coordinates in the x-axis.

$$\frac{\partial E}{\partial w}$$ times step is a small shift on $$f(w)$$ not $$w$$. So, why does the difference between $$W(n+1)$$ and $$W(n)$$ be equal to $$f(W)$$?

I found a similar question, but the accepted answer doesn't have a proof.

Don’t think about it as the $$w_{(n)}-w_{(n+1)}$$ being proportional to something. Think about it this way:

I'm now at $$w_{(n)}$$. Where do I want to be at timestep, so that the error decreases? For that, I need to know how the error changes when I make small steps to the left or right of $$w_{(n)}$$.

If $$E$$ increases as I increase $$w$$ (that is, if $$\frac{\partial E}{\partial w}>0$$, then obviously, I would want to move a little bit to the left. In other words, $$w_{n+1} or $$w_{n+1}-w_{n}<0$$.

On the other hand, if the derivative were negative, you know that you should move right to reduce the error a little bit, $$w_{n+1}-w_{n}>0$$. So, basically, your step should have the opposite sign of the derivative.

$$w_{n+1}-w_{n} \propto-\frac{\partial E}{\partial w}$$

$$\alpha$$, the learning rate, is just the constant of proportionality. Caution: think about small values for this rate, not big numbers. Taking a huge step can cause you to overshoot the minimum point.

$$let,\ x_{(n)} \ be\ a\ point\ on\ x-axis\ where\ f'( x) \ =\ 0\ ,\ and\ x_{(n\ +\ h)} \ is\ any\ other\ arbitary\ point \\ \therefore \ \ \frac{f'( x_{(n\ +\ h)})}{|\ f'( x_{(n\ +\ h)}) \ |} =\begin{cases} 1 & \mathrm{if,} \ h\ \ >0\\ 0 & \mathrm{if,} \ h\ =\ 0\\ -1 & \mathrm{if,} \ h\ < \ 0 \end{cases}\\similarly,\ \ \frac{x_{(n)} \ -\ x_{(n\ +\ h)} \ }{|\ x_{(n)} \ -\ x_{(n\ +\ h)} \ |} \ =\ \begin{cases} 1 & \mathrm{if,} \ h\ \ < 0\\ 0 & \mathrm{if,} \ h\ =\ 0\\ -1 & \mathrm{if,} \ h\ >\ 0 \end{cases}\\or,\ \frac{x_{(n)} \ -\ x_{(n\ +\ h)} \ }{|\ x_{(n)} \ -\ x_{(n\ +\ h)} \ |} \ =\ -\ \ \ \frac{f'( x_{(n\ +\ h)})}{|\ f'( x_{(n\ +\ h)}) \ |}\\ \therefore \ x_{(n)} \ =x_{(n\ +\ h)} \ -\ \eta \times f'( x_{(n\ +\ h)}) \ \ \ \ \ \ \left[ where\ \eta \ =\frac{|\ x_{(n)} \ -\ x_{(n\ +\ h)} \ |}{|f'( x_{(n\ +\ h)}) \ |} \ \right]$$

• I do not think you have proven anything here. In the equation second from bottom m and dy/dx cancel each out, right? What remains is x(n+h)-x(n)=h, which is kind of trivial, i.e. follows from notations you’ve chosen in (I) and (II). – aivanov Mar 21 '18 at 21:38