Share Your Experience: Take the 2024 Developer Survey

# Tag Info

## Hot answers tagged temporal-difference-methods

Accepted

### What algorithms are considered reinforcement learning algorithms?

The dynamic programming (DP) algorithms like policy iteration (PI) and value iteration (VI) are often presented in the context of reinforcement learning (in particular, in the book Reinforcement ...
• 40.8k
Accepted

### How to show temporal difference methods converge to MLE?

The convergence and optimality proofs of (linear) temporal-difference methods (under batch training, so not online learning) can be found in the paper Learning to predict by the methods of temporal ...
• 40.8k
Accepted

• 32.4k

### What algorithms are considered reinforcement learning algorithms?

In Reinforcement Learning: An Introduction the authors suggest that the topic of reinforcement learning covers analysis and solutions to problems that can be framed in this way: Reinforcement ...
• 32.4k
Accepted

### What is the intuition behind the TD(0) equation with average reward, and how is it derived?

This is simply from definition of return in average reward setting (look at equation $10.9$). The "standard" TD error is defined as TD_{\text{error}} = R_{t+1} + V(S_{t+1}) - V(S_t) \...
• 2,406
Accepted

### How fast does Monte Carlo tree search converge?

Yes, Monte Carlo tree search (MCTS) has been proven to converge to optimal solutions, under assumptions of infinite memory and computation time. That is, at least for the case of perfect-information, ...
• 10.3k
Accepted

### Why am I getting the incorrect value of lambda?

$TD(\lambda)$ return has the following form: $$G_t^\lambda = (1 - \lambda) \sum_{n=1}^{\infty} \lambda^{n-1} G_{t:t+n}$$ For you MDP $TD(1)$ looks like this: \begin{align} ...
• 2,406