3
$\begingroup$

Some examples of low-variance Machine Learning algorithms include Linear Regression, Linear Discriminant Analysis and Logistic Regression.

Examples of high-variance Machine Learning algorithms include Decision Trees, k-Nearest Neighbors and Support Vector Machines.

Source:

What makes a Machine Learning algorithm a low variance one or a high variance one?

KNN model

linear model

$\endgroup$
3
$\begingroup$

What this is talking about is how much a machine learning algorithm is good at "memorizing" the data. Decision trees, for their nature, tend to overfit very easily, this is because they can separate the space along very non-linear curves, especially if you get a very deep tree. Simpler algorithms, on the other hand, tend to separate the space along linear hyper surfaces, and therefore tend to under-fit the data and may not give very good prediction, but may behave better on new unseen data which is very different from the training data.

$\endgroup$
  • $\begingroup$ As mean and variance are independent of the different ML algorithms, the variance should be the same acc. to its formula. $\endgroup$ – Posi2 Jan 19 '19 at 1:31
  • $\begingroup$ Right. Over-fitting. $\endgroup$ – han_nah_han_ Jan 19 '19 at 5:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.