# Tag Info

## Hot answers tagged l1-regularization

3

In L1 regularization, the penalty term you compute for every parameter is a function of the absolute value of a given weight (times some regularization factor). Thus, irrespective of whether a weight is positive or negative (due to the absolute value) and irrespective of how large the weight is, there will be a penalty incurred as long as weight is unequal 0....

3

The following graph shows the constraint region (green), along with contours for Residual sum of squares (red ellipse). These are iso-lines signifying that points on an ellipse have the same RSS. Figure: Lasso (left) and Ridge (right) Constraints [Source: Elements of Statistical Learning] As Ridge regression has a circular constraint ($\beta_1^2 + \beta_2^... 1 The answer is largely the same whether we consider$\ell_1$or$\ell_2$regularisation, so I will just speak generally about regularisation. Mean square error for training data Given some training data$\{(x_i, y_i)\}_{i = 1}^n$, a linear regression line$Y = aX + b\$ fit using the least squares method looks for coefficients that minimise the sum of squares, ...

Only top voted, non community-wiki answers of a minimum length are eligible