Questions tagged [value-iteration]
For questions related to the value iteration algorithm, which is a dynamic programming (DP) algorithm used to solve an MDP, that is, it is used to find a policy given the transition and reward functions of the MDP. Value iteration is related to another DP algorithm called policy iteration.
5
questions
1
vote
2
answers
2k
views
Why and how can the policy and value iteration methods converge to the OPTIMAL point?
I am reading Reinforcement Learning: An Introduction by Sutton & Barto. According to this textbook, as far as I understood, the authors claim that the policy and value iteration methods converge ...
5
votes
1
answer
3k
views
Should the reward or the Q value be clipped for reinforcement learning
When extending reinforcement learning to the continuous states, continuous action case, we must use function approximators (linear or non-linear) to approximate the Q-value. It is well known that non-...
5
votes
0
answers
73
views
What exactly is non-delusional Q-learning?
Problems occur when we combine Q-learning with a function approximator.
What exactly is the delusional-bias and non-delusional Q-learning? I am talking about the neurIPS 18 best paper Non-delusional Q-...
3
votes
1
answer
2k
views
What is generalized policy iteration?
I am reading Sutton and Barto's material now. I know value iteration, which is an iterative algorithm taking the maximum value of adjacent states, and policy iteration. But what is generalized policy ...
1
vote
1
answer
835
views
In tic-tac-toe, what is the effect of the starting state on the state and action value function?
I am simulating a Tic-Tac-Toe game with a human opponent. The way the RL trains is through policy/value iterations for a fixed number of iterations all specified by the user. Now, whether the human ...