0
$\begingroup$

I am trying to better understand the Least Mean Squares algorithm, in order to implement it programmatically.

If we consider its weight updating formula $$w(n+1) = w(n) + \rho * \text{error}(i)x(i),$$ where $w(n + 1)$ is the new weight of the classifier function, $w(n)$ is its current weight and $x(i)$ is the $i$th element of a training dataset, what should $\rho$ be?

From what I have found online, $ρ$ is supposed to be $0 < \rho < \frac{2}{trace(X^TX)}$, where $X$ is a matrix with all the training data the algorithm has processed at that point. One idea that I had, was to take $\rho = \frac{1}{trace(X^TX)} < \frac{2}{trace(X^TX)}$, but I do not know if that is correct. Also, one characteristic that this value has is that it changes with each iteration of the algorithm, as more samples are added to matrix $X$.

So, what is a good value for $\rho$? Should it change during the execution of the algorithm or should it stay the same?

Improve this question
$\endgroup$
2
  • $\begingroup$ Hello. To clarify which algorithm you're referring to, are you talking about any of the ones mentioned here en.wikipedia.org/wiki/Least_mean_squares_filter? $\endgroup$
    – nbro
    Sep 6, 2021 at 11:15
  • $\begingroup$ @nbro Hi. Yes, the formula that I am using can be found in the section "LMS algorithm summary" of that article, although with different symbolism. $\endgroup$
    – User9123
    Sep 6, 2021 at 20:03

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .