# Tag Info

I'm going to use slightly different notation, $\leftarrow$ for an assignment, $\alpha$ for learning rate, $\nabla_w J$ in place of $g$* and implied multiplication as these are slightly more common. Also, using bold letters to represent vectors. In that notation, the update rule for basic gradient descent would be written as: $$\mathbf{w} \leftarrow \mathbf{... 3 The first two equations are equivalent. The last equation can be equivalent if you scale \alpha appropriately. Equation 1 Consider the equation from the Stanford slide:$$ v_{t}=\rho v_{t-1}+\nabla f(x_{t-1}) \\ x_{t}=x_{t-1}-\alpha v_{t},  Let's evaluate the first few $v_t$ so that we can arrive at a closed form solution: \$v_0 = 0 \\ v_1 = \rho v_0 + ...