# Tag Info

Accepted

### What algorithms are considered reinforcement learning algorithms?

The dynamic programming algorithms (like policy iteration and value iteration) are often presented in the context of reinforcement learning (in particular, in the book Reinforcement Learning: An ...
• 37k
Accepted

### How to show temporal difference methods converge to MLE?

The convergence and optimality proofs of (linear) temporal-difference methods (under batch training, so not online learning) can be found in the paper Learning to predict by the methods of temporal ...
• 37k
Accepted

• 26.6k

### What algorithms are considered reinforcement learning algorithms?

In Reinforcement Learning: An Introduction the authors suggest that the topic of reinforcement learning covers analysis and solutions to problems that can be framed in this way: Reinforcement ...
• 26.6k
Accepted

### What is the intuition behind the TD(0) equation with average reward, and how is it derived?

This is simply from definition of return in average reward setting (look at equation $10.9$). The "standard" TD error is defined as \begin{equation} TD_{\text{error}} = R_{t+1} + V(S_{t+1}) - V(S_t) \...
• 2,286
Accepted

### How fast does Monte Carlo tree search converge?

Yes, Monte Carlo tree search (MCTS) has been proven to converge to optimal solutions, under assumptions of infinite memory and computation time. That is, at least for the case of perfect-information, ...
• 9,794
Accepted

### Why am I getting the incorrect value of lambda?

$TD(\lambda)$ return has the following form: \begin{equation} G_t^\lambda = (1 - \lambda) \sum_{n=1}^{\infty} \lambda^{n-1} G_{t:t+n} \end{equation} For you MDP $TD(1)$ looks like this: \begin{align} ...
• 2,286