# Tag Info

Accepted

### What algorithms are considered reinforcement learning algorithms?

The dynamic programming algorithms (like policy iteration and value iteration) are often presented in the context of reinforcement learning (in particular, in the book Reinforcement Learning: An ...
• 36.4k
Accepted

### How to show temporal difference methods converge to MLE?

The convergence and optimality proofs of (linear) temporal-difference methods (under batch training, so not online learning) can be found in the paper Learning to predict by the methods of temporal ...
• 36.4k
Accepted

• 25.5k

### What algorithms are considered reinforcement learning algorithms?

In Reinforcement Learning: An Introduction the authors suggest that the topic of reinforcement learning covers analysis and solutions to problems that can be framed in this way: Reinforcement ...
• 25.5k
Accepted

### What is the intuition behind TD($\lambda$)?

TD($\lambda$) can be thought of as a combination of TD and MC learning, so as to avoid to choose one method or the other and to take advantage of both approaches. More precisely, TD($\lambda$) is ...
• 36.4k
Accepted

### What is the intuition behind the TD(0) equation with average reward, and how is it derived?

This is simply from definition of return in average reward setting (look at equation $10.9$). The "standard" TD error is defined as TD_{\text{error}} = R_{t+1} + V(S_{t+1}) - V(S_t) \...
• 2,286
Accepted

### How fast does Monte Carlo tree search converge?

Yes, Monte Carlo tree search (MCTS) has been proven to converge to optimal solutions, under assumptions of infinite memory and computation time. That is, at least for the case of perfect-information, ...
• 9,659
Accepted

### Why am I getting the incorrect value of lambda?

$TD(\lambda)$ return has the following form: $$G_t^\lambda = (1 - \lambda) \sum_{n=1}^{\infty} \lambda^{n-1} G_{t:t+n}$$ For you MDP $TD(1)$ looks like this: \begin{align} ...
• 2,286