4

In machine learning, the term bias can refer to at least 2 related concepts A (learnable) parameter of a model, such as a linear regression model, which allows you to learn a shifted function. For example, in the case of a linear regression model $y = f(x) = mx + b$, the bias $b$ allows you to shift the straight-line up an down: without the bias, you would ...


3

Section 5.2 Error Decomposition of the book Understanding Machine Learning: From Theory to Algorithms (2014) gives a description of the approximation error and estimation error in the context of empirical risk minimization (ERM) and, in particular, in the context of the bias-complexity tradeoff (which is strictly related to the bias-variance tradeoff). Error/...


3

What this is talking about is how much a machine learning algorithm is good at "memorizing" the data. Decision trees, for their nature, tend to overfit very easily, this is because they can separate the space along very non-linear curves, especially if you get a very deep tree. Simpler algorithms, on the other hand, tend to separate the space along linear ...


2

The bias-variance trade-off that you're referring to has to do with the return estimator. Any RL algorithm you choose needs some estimate of the cumulative return, which is a random variable with many sources of randomness, such as stochastic transitions or rewards. Monte Carlo RL algorithms estimate returns by running full trajectories and literally ...


2

When using terms like "high" for high variance, this is in comparison to other methods, mainly in comparison to TD learning, which bootstraps between single time steps. It is worth spelling out what the variance applies to and where it comes from: Namely the Monte Carlo return $G_t$ distribution, which can be calculated as follows: $$G_t = \sum_{k=0}^{T-t-...


1

To gain a good understanding of this, I recommend first reading about the trade-off between bias and variance in ML and AI methods. A great article on this topic that I recommend as a light mathematical introduction is this: https://towardsdatascience.com/understanding-the-bias-variance-tradeoff-165e6942b229 In short: Bias represents the models effort to ...


1

Bias is not necessarily bad, even though the term bias usually has a negative connotation. In fact, in machine learning, inductive bias is quite important and necessary. For example, if you want to learn a function $f(x) = y$, where $x \in \mathcal{X}$ and $y \in \mathcal{Y}$, you often just have a finite dataset $\mathcal{D} = \{ (x_i, y_i)\}_{i=1}^N$, ...


1

Having low variance is important in general as it reduces the number of samples needed to obtain accurate estimates. This is the case for all statistical machine learning, not just reinforcement learning. In general, if you are estimating a mean or expected quantity by taking many samples, the variation in the error is proportional to $\frac{\sigma}{\sqrt{N}...


Only top voted, non community-wiki answers of a minimum length are eligible