# What should the value of $ρ$ in the $w(n+1) = w(n) + \rho*\text{error}(i)x(i)$ formula of Least Mean Squares be?

I am trying to better understand the Least Mean Squares algorithm, in order to implement it programmatically.

If we consider its weight updating formula $$w(n+1) = w(n) + \rho * \text{error}(i)x(i),$$ where $$w(n + 1)$$ is the new weight of the classifier function, $$w(n)$$ is its current weight and $$x(i)$$ is the $$i$$th element of a training dataset, what should $$\rho$$ be?

From what I have found online, $$ρ$$ is supposed to be $$0 < \rho < \frac{2}{trace(X^TX)}$$, where $$X$$ is a matrix with all the training data the algorithm has processed at that point. One idea that I had, was to take $$\rho = \frac{1}{trace(X^TX)} < \frac{2}{trace(X^TX)}$$, but I do not know if that is correct. Also, one characteristic that this value has is that it changes with each iteration of the algorithm, as more samples are added to matrix $$X$$.

So, what is a good value for $$\rho$$? Should it change during the execution of the algorithm or should it stay the same?

• Hello. To clarify which algorithm you're referring to, are you talking about any of the ones mentioned here en.wikipedia.org/wiki/Least_mean_squares_filter?
– nbro
Sep 6, 2021 at 11:15
• @nbro Hi. Yes, the formula that I am using can be found in the section "LMS algorithm summary" of that article, although with different symbolism. Sep 6, 2021 at 20:03