# Questions tagged [rl-an-introduction]

For questions related to the book "Reinforcement Learning: An Introduction" (by Andrew Barto and Richard S. Sutton).

25 questions
Filter by
Sorted by
Tagged with
35 views

### Doubt regarding the proof of convergence of $\epsilon$ soft policies without exploring starts

In page 125 of Sutton and Barto (second last paragraph) the proof for equality of $v_{\pi}$ and $v_*$ for $\epsilon$ soft policies is given. But I could not understand the statement explaining the ...
62 views

### What does the figure “Blackjack Value Function…” from Sutton represent?

I came across this graph in David Silver's youtube lecture and Sutton's book on reinforcement learning. Can anyone help me understand the graph? From the graph, for 10000 episodes what i see is ...
43 views

### On-policy state distribution for episodic tasks on Sutton & Barto, page 199

In Sutton & Barto's "Reinforcement Learning: An Introduction", 2nd edition, page 199, they describe the on-policy distribution for episodic tasks in the following box: I don't understand how this ...
70 views

### Counterexamples to the reward hypothesis

On Sutton and Barto's RL book, the reward hypothesis is stated as that all of what we mean by goals and purposes can be well thought of as the maximization of the expected value of the cumulative ...
49 views

83 views

### Hashed Tile Coding vs Regular Tile Coding

In the book "Reinforcement Learning: An Introduction" (2018) Sutton and Barto explain at page 221 a form of tile coding using hashing, to reduce memory consumption. I have two questions about that: ...
57 views

### Possible inconsistency in the Policy Improvement equation

I came across this formula in Sutton And Barto: RL an Intro (2nd Edition) equation number 4.7 (page number 78). If $\pi$ and $\pi'$ are deterministic policies and $q_\pi(s, \pi'(s)) \geq v_\pi(s)$ ...
297 views

### How is the policy gradient calculated in REINFORCE?

Reading Sutton and Barto, I see the following in describing policy gradients: How is the gradient calculated with respect to an action (taken at time t)? I've read implementations of the algorithm, ...
197 views

65 views

### What is the meaning of Model(s, a) in the prioritized sweeping algorithm?

I'm reading the book "Reinforcement Learning: An Introduction" (by Andrew Barto and Richard S. Sutton). The authors provide the pseudocode of the prioritized sweeping algorithm, but I do not know ...
815 views

### How do we prove the n-step return error reduction property?

In section 7.1 (about the n-step bootstrapping) of the book Reinforcement Learning: An Introduction (2nd edition), by Andrew Barto and Richard S. Sutton, the authors write about what they call the "n-...
113 views

### Understanding the notation in the definition of the expected reward

I am new to RL and I am trying to work through the book Reinforcement Learning: An Introduction I (Sutton & Barto, 2018). In chapter 3 on Finite Markov Decision Processes, the authors write the ...
63 views

### Is my interpretation of the return correct?

Sutton and Barto 2018 define the discounted return $G_t$ the following way (p 55): Is my interpretation correct? Or should all "1" be in the same column?