Questions tagged [policy-evaluation]
For questions related to the various policy evaluation (PE) algorithms, which are numerical iterative algorithms that are used to find the value function associated with a given policy, which is often denoted as the "prediction problem". PE is also considered a dynamic programming method, which is regularly discussed in reinforcement learning textbooks.
3
questions
21
votes
2
answers
20k
views
What is the difference between First-Visit Monte-Carlo and Every-Visit Monte-Carlo Policy Evaluation?
I came across these 2 algorithms, but I cannot understand the difference between these 2, both in terms of implementation as well as intuitionally.
So, what difference does the second point in both ...
user9947
8
votes
2
answers
4k
views
What is the proof that policy evaluation converges to the optimal solution?
Although I know how the algorithm of iterative policy evaluation using dynamic programming works, I am having a hard time realizing how it actually converges.
It appeals to intuition that, with each ...
- 107
4
votes
3
answers
845
views
Why can the Bellman equation be turned into an update rule?
In chapter 4.1 of Sutton's book, the Bellman equation is turned into an update rule by simply changing the indices of it. How is it mathematically justified? I didn't quite get the initiation of why ...
- 123