3
votes
Accepted
Which is a better form of regularization: lasso (L1) or ridge (L2)?
The following graph shows the constraint region (green), along with contours for Residual sum of squares (red ellipse). These are iso-lines signifying that points on an ellipse have the same RSS.
...
- 370
3
votes
Accepted
Why does L1 regularization yield sparse features?
In L1 regularization, the penalty term you compute for every parameter is a function of the absolute value of a given weight (times some regularization factor).
Thus, irrespective of whether a weight ...
- 825
2
votes
Does L1/L2 Regularization help reach an optimum result faster?
I am not aware of any empirical results regarding this question.
But in theory, adding a regularization term shall make the learning task actually even harder, since there is suddenly a second loss ...
- 825
2
votes
Accepted
What are the consequences when we multiply, instead of add, a penalty term?
Well let's consider one, Ridge regression.
We have 2 terms:
the regression loss $L^{pred} = \sum(f(x) - y)^2$, which we can see that it is a sum of squared values, thus $L^{pred} \ge 0$
the ...
- 2,632
1
vote
Accepted
Would either $L_1$ or $L_2$ regularisation lower the MSE on the training and test data?
The answer is largely the same whether we consider $\ell_1$ or $\ell_2$ regularisation, so I will just speak generally about regularisation.
Mean square error for training data
Given some training ...
- 1,010
Only top scored, non community-wiki answers of a minimum length are eligible