0
$\begingroup$

https://en.wikipedia.org/wiki/ADALINE

Can Adaline(Adaptive Linear Neuron) be used to do a multiple linear regression being equivalent to the least squares method?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Though conceptually they're very similar, your own reference points out their essential difference:

This update rule minimizes $E$, the square of the error, and is in fact the stochastic gradient descent update for linear regression.

Thus Adaline usually as a delta training rule for a (single) linear output layer in a ANN updates the model's weights incrementally based on individual training samples, which can lead to faster convergence of its error function and is suitable for online learning scenarios where data is continuously streaming. While the traditional OLS closed-form analytical approach for multiple linear regression in statistics requires all sample in a single batch.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks!. I have other doubt about your answer: Does the fact that traditional OLS uses a single batch, and Adaline based on individual training samples, affect the accuracy or the final result? or should both produce similar results as well? please explain more about the batch? $\endgroup$
    – will The J
    Commented Nov 4, 2023 at 23:42
  • $\begingroup$ Of course OLS is more accurate once you gather all data inputs provided it has its closed form solution relative to SGD and even full GD. It's well known GD could be very slow and may converge to local minimum while OLS attains global minimum of squared error. $\endgroup$
    – cinch
    Commented Nov 4, 2023 at 23:47
  • $\begingroup$ Thanks!. Does this mean that although Adaline can be used for multiple linear regression, it runs the risk of getting stuck in a local minimum during training? while OLS doesn't run this risk, and can be more accurate in some cases? I am correct? $\endgroup$
    – will The J
    Commented Nov 4, 2023 at 23:53
  • $\begingroup$ Yes that’s certainly a reasonable conclusion and historically the backpropagation algo based upon gradient descent for MLPs was inspired by the Adaline SGD, not OLS. $\endgroup$
    – cinch
    Commented Nov 5, 2023 at 0:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .