4

In machine learning, the term bias can refer to at least 2 related concepts A (learnable) parameter of a model, such as a linear regression model, which allows you to learn a shifted function. For example, in the case of a linear regression model $y = f(x) = mx + b$, the bias $b$ allows you to shift the straight-line up an down: without the bias, you would ...


3

What this is talking about is how much a machine learning algorithm is good at "memorizing" the data. Decision trees, for their nature, tend to overfit very easily, this is because they can separate the space along very non-linear curves, especially if you get a very deep tree. Simpler algorithms, on the other hand, tend to separate the space along linear ...


2

The bias-variance trade-off that you're referring to has to do with the return estimator. Any RL algorithm you choose needs some estimate of the cumulative return, which is a random variable with many sources of randomness, such as stochastic transitions or rewards. Monte Carlo RL algorithms estimate returns by running full trajectories and literally ...


2

When using terms like "high" for high variance, this is in comparison to other methods, mainly in comparison to TD learning, which bootstraps between single time steps. It is worth spelling out what the variance applies to and where it comes from: Namely the Monte Carlo return $G_t$ distribution, which can be calculated as follows: $$G_t = \sum_{k=0}^{T-t-...


2

It is not so much the problem of using Reinforcement Learning to train the neural networks, it is the assumptions made about the data given to standard Neural Networks. They are not capable of handling strongly correlated data which is one of the motivations for introducing Recurrent Neural Networks, as they can handle this correlated data well.


1

If you read the relevant section. it also says: Model compression is applicable when the size of the original model is driven primarily by a need to prevent overfitting. In most cases, the model with the lowest generalization error is an ensemble of several independently trained models. Evaluating all $n$ ensemble members is expensive. Sometimes, even a ...


1

To gain a good understanding of this, I recommend first reading about the trade-off between bias and variance in ML and AI methods. A great article on this topic that I recommend as a light mathematical introduction is this: https://towardsdatascience.com/understanding-the-bias-variance-tradeoff-165e6942b229 In short: Bias represents the models effort to ...


1

Bias is not necessarily bad, even though the term bias usually has a negative connotation. In fact, in machine learning, inductive bias is quite important and necessary. For example, if you want to learn a function $f(x) = y$, where $x \in \mathcal{X}$ and $y \in \mathcal{Y}$, you often just have a finite dataset $\mathcal{D} = \{ (x_i, y_i)\}_{i=1}^N$, ...


1

Having low variance is important in general as it reduces the number of samples needed to obtain accurate estimates. This is the case for all statistical machine learning, not just reinforcement learning. In general, if you are estimating a mean or expected quantity by taking many samples, the variation in the error is proportional to $\frac{\sigma}{\sqrt{N}...


Only top voted, non community-wiki answers of a minimum length are eligible