Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.

# Questions tagged [regularization]

For questions about application of regularization techniques.

51 questions
Filter by
Sorted by
Tagged with
17 views

### Why does loss function and constraint touches at corner point in lasso regression?

As you can see in the picture, for two co-efficient w1 and w2, our loss function, f(w1, w2) should be minimized under constraint function g(w1, w2). For lasso regularization minimum point always lies ...
38 views

### Can we Consider Regularization as a "Constraint"?

I have the following question on "Regularization vs. Constrained Optimization" : In the context of statistical modelling, we are often taught about "Regularization" as a method of ...
• 287
155 views

### How does Regularization Reduce Overfitting?

As I understand, this is the general summary of the Regularization-Overfitting Problem: The classical "Bias-Variance Tradeoff" suggests that complicated models (i.e. models with more ...
• 287
50 views

### What determines when Dropout, BatchNorm & other Regularization will be effective?

I just had a very strange experience where I was training an 8 layer deep & pretty wide (max: 512 neurons) neural network for a regression task. I had assumed since it was big enough that it would ...
• 266
17 views

• 21
69 views

### Is there a way to ensure that my model is able to recognize an unseen example?

My question is more theoretical than practical. Let's say that I am training my cat classifier with a dataset that I feel is pretty representative of cat images in general. But then a new breed of cat ...
• 41
426 views

### Are there any rules of thumb for having some idea of what capacity a neural network needs to have for a given problem?

To give an example. Let's just consider the MNIST dataset of handwritten digits. Here are some things which might have an impact on the optimum model capacity: There are 10 output classes The inputs ...
• 1,089
1 vote
112 views

### What is the difference between TensorFlow's callbacks and early stopping?

What is the difference between TensorFlow's callbacks and early stopping?
• 47
124 views

### How do I poison an SVM with manifold regularization?

I'm working on Adversarial Machine Learning, and have read multiple papers on this topic, some of them are mentioned as follows: Poisoning Attacks on SVMs: https://arxiv.org/pdf/1206.6389.pdf ...
• 115
99 views

### Which is a better form of regularization: lasso (L1) or ridge (L2)?

Given a ridge and a lasso regularizer, which one should be chosen for better performance? An intuitive graphical explanation (intersection of the elliptical contours of the loss function with the ...
• 188
1 vote