Skip to main content

# Tag Info

## Hot answers tagged policy-improvement

3 votes

### Why do we need to go back to policy evaluation after policy improvement if the policy is not stable?

There is a difference between accurate value function estimates, and optimal value functions. An optimal value function is more specifically the value function of an optimal policy. Value functions ...
• 32.7k
3 votes
Accepted

### Is value iteration stopped after one update of each state?

Where the author mentions the policy evaluation being stopped after one state, they are referring to the part of the algorithm that evaluates the policy -- the pseudocode you have listed is the ...
• 4,920
2 votes
Accepted

### Understanding the update rule for the policy in the policy iteration algorithm

A policy can be stochastic or deterministic. A deterministic policy is a function of the form $\pi_{\text{deterministic}}: S \rightarrow A$, that is, a function from the set of states to the set of ...
• 40.9k
2 votes
Accepted

### How is the first line obtained and where is the information $v_{\pi}(x_k)=v_{\pi'}(x_k)$ used in the following derivation regarding greedy policy?

Policy iteration means to improve the policy step by step, so that $v_{\pi}(x_{k}) \leq v_{\pi'}(x_{k})$ for all $x \in X$. If we have $v_{\pi}(x_{k}) = v_{\pi'}(x_{k})$ for all $x \in X$, this means ...
• 332
2 votes
Accepted

### How do we get from conditional expectation on both state and action to only state in the proof of the Policy Improvement Theorem?

I don't understand how did we get rid of the condition $A_{t}=\pi'(s)$. We don't really, it is just moved into the subscript $\pi'$ in $\mathbb{E}_{\pi'}[]$ - it means the same thing here, that the ...
• 32.7k
1 vote
Accepted

### Monte Carlo epsilon-greedy Policy Iteration: monotonic improvement for all cases or for the expected value?

I think this equation answer your question: $$q_{\pi^{i}}(s,\pi^{i+1}(s)) = \mathbf{E}[q_{\pi^{i}}(s,\pi^{i+1}(s))] = \sum_{a \in A}\pi^{i+1}(a|s)q_{\pi^{i}}(s,a)$$ value of the Q while taking ...
• 693

Only top scored, non community-wiki answers of a minimum length are eligible