Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [bootstrapping]

For questions related to the concept of bootstrapping in reinforcement learning (and statistics).

Filter by
Sorted by
Tagged with
1 vote
1 answer
225 views

Doesn't the n-step Tree Backup algorithm negatively affect the DQN-Agent by creating inconsistent look-ahead targets?

In the text book of Sutton and Barto on page 152 they introduce the n-step Tree Backup algorithm, where the tree-backup n-step return is defined via $$ G_{t:t+n} = R_{t+1} + \gamma \sum_{a \neq A_{t+1}...
Peter's user avatar
  • 55
1 vote
1 answer
597 views

How is $Q(s', a')$ calculated in SARSA and Q-Learning?

I have a question about how to update the Q-function in Q-learning and SARSA. Here (What are the differences between SARSA and Q-learning?) the following updating formulas are given: Q-Learning $$Q(s,...
PeterBe's user avatar
  • 256
3 votes
0 answers
201 views

How does bootstrapping work with the offline $\lambda$-return algorithm?

In Barton and Sutton's book, Reinforcement Learning: An Introduction (2nd edition), an expression, on page 289 (equation 12.2), introduced the form of the $\lambda$-return defined as follows $$G_t^{\...
quest ions's user avatar
1 vote
0 answers
115 views

Why do bootstrapping methods produce nonstationary targets more than non-bootstrapping methods?

The following quote is taken from the beginning of the chapter on "Approximate Solution Methods" (p. 198) in "Reinforcement Learning" by Sutton & Barto (2018): reinforcement ...
Johan's user avatar
  • 121