Some examples of low-variance machine learning algorithms include linear regression, linear discriminant analysis, and logistic regression.
Examples of high-variance machine learning algorithms include decision trees, k-nearest neighbors, and support vector machines.
Source:
What makes a machine learning algorithm a low variance one or a high variance one? For example, why do decision trees, k-NNs and SVMs have high variance?
What this is talking about is how much a machine learning algorithm is good at "memorizing" the data. Decision trees, for their nature, tend to overfit very easily, this is because they can separate the space along very non-linear curves, especially if you get a very deep tree. Simpler algorithms, on the other hand, tend to separate the space along linear hyper surfaces, and therefore tend to under-fit the data and may not give very good prediction, but may behave better on new unseen data which is very different from the training data.
An algorithm's bias and variance can be thought of as its property, this can be tweaked with things that we call as hyperparameters, but every algorithm has its own set of assumptions that it makes which if fulfilled, the algorithm performs better.
Some algorithms such as Logistic Regression, linear-SVMs (not the kernel SVMs, because they can be used for non-linear problems as well) etc are linear models and work well if the data is linearly separable. If the data can not be separated by a linear plane, then no matter how much you tweak and fine-tune them, it won't work, because the data simply can not be separated by a linear plane, and that is the bias everyone talks about for these kinds of algorithms.
On the other hand, Decision Trees can split the whole space into several hypercubes and based on which hypercube a datapoint is in, they classify that datapoint. KNNs on the other hand, use the neighbours of a datapoint and their types/properties to make predictions. Thus, a change in the positions of those datapoints will largely affect both the decision boundary(s) of both of these algorithms, and that is why they can be very easily overfitted and have a high variance.
Hope this helps.
