# what is the proof behind the gradient of a curve being equal/proportional to the distance between the two co-ordinates in the x-axis [closed]

In the delta rule the equation to adjust the weight with respect to error is :-

where is the Learning Rate and E is the Error

The graph for E vs w would look like the one below with E in the y axis and W in the x axis

In other words we can write

I want to know, what is the proof behind the gradient of a curve being equal/proportional to the distance between the two co-ordinates in the x-axis

-OR-

(∂E/∂w) times step is a small shift on f(w) not w.so why does the difference between W(n+1) and Wn be equal to f(W)

I found a similar question some-confusion-of-gradient-descent, but the accepted answer doesnot have a proof.

• Welcome to AI.SE! I'm sorry, this site is for social/conceptual/academic aspects of artificial intelligence as opposed to mathematical/implementation issues. For AI-related math, Cross Validated might be able to help; for pure math, Mathematics is the place to go. For more info on our site, see the tour. – Ben N Jan 22 '18 at 22:25

Don’t think about it as the being proportional to something. Think about it this way:

I’m now at . Where do I want to be at Time step so that the error decreases? For that, I need to know how the error changes when I make small steps to the left or right of If increases as I increase (that is, if , then obviously, I would want to move a little bit to the left. In other words, or .

On the other hand, if the derivative were negative, you know that you should move right to reduce the error a little bit, . So, basically your step should have the opposite sign of the derivative.

, the learning rate, is just the constant of proportionality. Caution: think about small values for this rate, not big numbers. Taking a huge step can cause you to overshoot the minimum point.

• Oops, no formatting, unlike Math SE. I hope you are able to understand, though. – Mathemagical Jan 21 '18 at 11:42
• i have updated the question while you were answering, can you just it once more, TIA – souparno majumder Jan 21 '18 at 11:43
• that brings me to another question, if changing the w changes the error, then why cant i write wn - wn+1 be proportional to E? insted why do i need a derivative of E? – souparno majumder Jan 21 '18 at 12:10
• @souparnomajumder proportional to E will not work. The bigger the error, you want to move more to the right? Imagine yourself at w=2 on your graph. Moving to the right will give you even more error! – Mathemagical Jan 21 '18 at 13:09
• can you check if the following derivation is correct ? mathcha.io/editor/8NmjHWQUxztpofYO – souparno majumder Jan 21 '18 at 14:43

$$let,\ x_{(n)} \ be\ a\ point\ on\ x-axis\ where\ f'( x) \ =\ 0\ ,\ and\ x_{(n\ +\ h)} \ is\ any\ other\ arbitary\ point \\ \therefore \ \ \frac{f'( x_{(n\ +\ h)})}{|\ f'( x_{(n\ +\ h)}) \ |} =\begin{cases} 1 & \mathrm{if,} \ h\ \ >0\\ 0 & \mathrm{if,} \ h\ =\ 0\\ -1 & \mathrm{if,} \ h\ < \ 0 \end{cases}\\similarly,\ \ \frac{x_{(n)} \ -\ x_{(n\ +\ h)} \ }{|\ x_{(n)} \ -\ x_{(n\ +\ h)} \ |} \ =\ \begin{cases} 1 & \mathrm{if,} \ h\ \ < 0\\ 0 & \mathrm{if,} \ h\ =\ 0\\ -1 & \mathrm{if,} \ h\ >\ 0 \end{cases}\\or,\ \frac{x_{(n)} \ -\ x_{(n\ +\ h)} \ }{|\ x_{(n)} \ -\ x_{(n\ +\ h)} \ |} \ =\ -\ \ \ \frac{f'( x_{(n\ +\ h)})}{|\ f'( x_{(n\ +\ h)}) \ |}\\ \therefore \ x_{(n)} \ =x_{(n\ +\ h)} \ -\ \eta \times f'( x_{(n\ +\ h)}) \ \ \ \ \ \ \left[ where\ \eta \ =\frac{|\ x_{(n)} \ -\ x_{(n\ +\ h)} \ |}{|f'( x_{(n\ +\ h)}) \ |} \ \right]$$

• I do not think you have proven anything here. In the equation second from bottom m and dy/dx cancel each out, right? What remains is x(n+h)-x(n)=h, which is kind of trivial, i.e. follows from notations you’ve chosen in (I) and (II). – aivanov Mar 21 '18 at 21:38