Questions tagged [bias-variance-tradeoff]

For questions related to the bias-variance tradeoff, which is an important issue in machine learning.

Filter by
Sorted by
Tagged with
1 vote
1 answer
82 views

How does deep learning overcome overfitting?

From Berkeley CS182, SP22: https://cs182sp22.github.io/assets/lecture_slides/2022.01.26-ml-review-pt2.pdf. Can someone help me interpret this diagram? I understand the graph on the left, but I don't ...
9j09jf02jsd's user avatar
0 votes
0 answers
28 views

Effectiveness of DNN training with reduced Batch randomness

So here's an example set to help explain my doubt. Suppose I have 80,000 total images available for a DNN training task. With a batch size of 32, that is 2500 batches. Now let's say I partition the ...
Keshav Vinayak Jha's user avatar
0 votes
1 answer
46 views

How to evaluate binary classifier on imbalanced dataset?

I have trained a Decision Tree model on an imbalanced dataset. I got the following results for the test set from the sklearn and imblearn classification reports (attached below). Moreover, the other ...
Zal's user avatar
  • 7
0 votes
2 answers
708 views

Why does k-means have more bias than spectral clustering and GMM?

I ran into a 2019-Entrance Exam question as follows: The answer mentioned is (4), but some search on google showed me maybe (1) and (2) is equal to (4). Why would k-means be the algorithm with the ...
Lisa Berry's user avatar
1 vote
0 answers
62 views

Bias-variance tradeoff and learning curves for non-deep learning models

I am following a course on machine learning and am confused about the bias-variance trade-off relationship to learning curves in classification. I am seeing some conflicting information online on this....
ML-Student-1996's user avatar
3 votes
1 answer
3k views

What's the difference between estimation and approximation error?

I'm unable to find online, or understand from context - the difference between estimation error and approximation error in the context of machine learning (and, specifically, reinforcement learning). ...
esoteric-elliptic's user avatar
2 votes
0 answers
92 views

Why don't ensembling, bagging and boosting help to improve accuracy of Naive bayes classifier?

You might think to apply some classifier combination techniques like ensembling, bagging and boosting but these methods would not help. Actually, “ensembling, boosting, bagging” won’t help since their ...
Sivaram Rasathurai's user avatar
7 votes
1 answer
2k views

Is there a connection between the bias term in a linear regression model and the bias that can lead to under-fitting?

Here is a linear regression model $$y = mx + b,$$ where $b$ is known as $y$-intercept, but also known as the bias [1], $m$ is the slope, and $x$ is the feature vector. As I understood, in machine ...
Sivaram Rasathurai's user avatar
2 votes
1 answer
84 views

Why are large models necessary when we have a limited number of training examples?

In Goodfellow et al. book Deep Learning chapter 12.1.4 they write These large models learn some function $f(x)$, but do so using many more parameters than are necessary for the task. Their size is ...
Borun Chowdhury's user avatar
2 votes
1 answer
1k views

What is the bias-variance trade-off in reinforcement learning?

I am watching DeepMind's video lecture series on reinforcement learning, and when I was watching the video of model-free RL, the instructor said the Monte Carlo methods have less bias than temporal-...
Aman Savaria's user avatar
1 vote
0 answers
33 views

Do the variance and bias belong to the policy or value functions?

Recently, I read many papers on variance and bias. But I am still confused by the two notions, the variance or bias belongs to who? Policy or value? If the variance or bias is large or low, what ...
GoingMyWay's user avatar
3 votes
1 answer
824 views

How does Monte Carlo have high variance?

I was going through David Silver's lecture on reinforcement learning (lecture 4). At 51:22 he says that Monte Carlo (MC) methods have high variance and zero bias. I understand the zero bias part. It ...
Bhuwan Bhatt's user avatar
2 votes
1 answer
81 views

How can I determine the bias and variance of a random forrest?

On this website https://scikit-learn.org/stable/modules/learning_curve.html, the authors are speaking about variance and bias and they give a simple example of how works in a linear model. How can I ...
jennifer ruurs's user avatar
4 votes
2 answers
372 views

Why is having low variance important in offline policy evaluation of reinforcement learning?

Intuitively, I understand that having an unbiased estimate of a policy is important because being biased just means that our estimate is distant from the truth value. However, I don't understand ...
Hunnam 's user avatar
  • 227
2 votes
0 answers
206 views

How is the bias caused by a max pooling layer overcome?

I have constructed a CNN that utilizes max-pooling layers. I have found with these layers that, should I remove them, my network performs ideally with every output and gradient at each layer having a ...
Recessive's user avatar
  • 1,326
3 votes
2 answers
447 views

What makes a machine learning algorithm a low variance one or a high variance one?

Some examples of low-variance machine learning algorithms include linear regression, linear discriminant analysis, and logistic regression. Examples of high-variance machine learning algorithms ...
Posi2's user avatar
  • 338
4 votes
1 answer
122 views

Can I compute the fitness of an agent based on a low number of runs of the game?

I'm developing an AI to play a card game with a genetic algorithm. Initially, I will evaluate it against a player that plays randomly, so there will naturally be a lot of variance in the results. I ...
Ryxuma's user avatar
  • 237