Questions tagged [bias-variance-tradeoff]
For questions related to the bias-variance tradeoff, which is an important issue in machine learning.
11
questions
2
votes
0answers
19 views
Why don't ensembling, bagging and boosting help to improve accuracy of Naive bayes classifier?
You might think to apply some classifier combination techniques like ensembling, bagging and boosting but these methods would not help. Actually, “ensembling, boosting, bagging” won’t help since their ...
2
votes
1answer
59 views
Is there a connection between the bias term in a linear regression model and the bias that can lead to under-fitting?
Here is a linear regression model
$$y = mx + b,$$
where $b$ is known as $y$-intercept, but also known as the bias [1], $m$ is the slope, and $x$ is the feature vector.
As I understood, in machine ...
2
votes
1answer
51 views
Why are large models necessary when we have a limited number of training examples?
In Goodfellow et al. book Deep Learning chapter 12.1.4 they write
These large models learn some function $f(x)$, but do so using many more parameters than are necessary for the task. Their size is ...
2
votes
1answer
74 views
What is the bias-variance trade-off in reinforcement learning?
I am watching DeepMind's video lecture series on reinforcement learning, and when I was watching the video of model-free RL, the instructor said the Monte Carlo methods have less bias than temporal-...
1
vote
0answers
25 views
Do the variance and bias belong to the policy or value functions?
Recently, I read many papers on variance and bias. But I am still confused by the two notions, the variance or bias belongs to who? Policy or value? If the variance or bias is large or low, what ...
3
votes
1answer
115 views
How does Monte Carlo have high variance?
I was going through David Silver's lecture on reinforcement learning (lecture 4). At 51:22 he says that Monte Carlo (MC) methods have high variance and zero bias. I understand the zero bias part. It ...
7
votes
1answer
103 views
Why does reinforcement learning using a non-linear function approximator diverge when using strongly correlated data as input?
While reading the DQN paper, I found that randomly selecting and learning samples reduced divergence in RL using a non-linear function approximator (e.g a neural network).
So why does Reinforcement ...
2
votes
1answer
45 views
How can I determine the bias and variance of a random forrest?
On this website https://scikit-learn.org/stable/modules/learning_curve.html, the authors are speaking about variance and bias and they give a simple example of how works in a linear model.
How can I ...
4
votes
2answers
81 views
Why is having low variance important in offline policy evaluation of reinforcement learning?
Intuitively, I understand that having an unbiased estimate of a policy is important because being biased just means that our estimate is distant from the truth value.
However, I don't understand ...
2
votes
0answers
70 views
How is the bias caused by a max pooling layer overcome?
I have constructed a CNN that utilizes max-pooling layers. I have found with these layers that, should I remove them, my network performs ideally with every output and gradient at each layer having a ...
3
votes
2answers
179 views
What makes a machine learning algorithm a low variance one or a high variance one?
Some examples of low-variance machine learning algorithms include linear regression, linear discriminant analysis, and logistic regression.
Examples of high-variance machine learning algorithms ...
