# Questions tagged [policy-improvement-theorem]

For questions related to the policy improvement theorem in the context of reinforcement learning.

9 questions
Filter by
Sorted by
Tagged with
1 vote
20 views

### Understanding the policy improvement theorem for Monte Carlo Control without Exploring Starts

I am currently studying the equations 5.2 in Reinforcement Learning An Introduction By Sutton and Barto on page 101. I want to comprehent the proof by a simple example: Having only one State with two ...
• 13
188 views

### Why policy improvement theorem can't be applied in case of function approximation?

Policy improvement theorem is proven as follows: $$v_\pi(s) = \sum_{a \in A} \pi(a|s)q_\pi(a,s) \leq \max_{a \in A} q_\pi(a,s) = q_\pi(\pi'(s), s)$$ What step of the proof does not hold for function ...
120 views

### Could you explain steps of proofs of policy improvement theorem in sutton & barto book?

I don't understand one step of the proof. The equality means $\mathop{\mathbb{E}}[R_{t+2}+\gamma v_\pi(S_{t+2})|S_{t+1},A_{t+1}=\pi'(S_{t+1})]=R_{t+2}+\gamma v_\pi(S_{t+2})$ Could you explain the ...
• 205
1 vote
2k views

### Why and how can the policy and value iteration methods converge to the OPTIMAL point?

I am reading Reinforcement Learning: An Introduction by Sutton & Barto. According to this textbook, as far as I understood, the authors claim that the policy and value iteration methods converge ...
• 113