Questions tagged [value-iteration]

For questions related to the value iteration algorithm, which is a dynamic programming (DP) algorithm used to solve an MDP, that is, it is used to find a policy given the transition and reward functions of the MDP. Value iteration is related to another DP algorithm called policy iteration.

Filter by
Sorted by
Tagged with
1
vote
0answers
48 views

Is Value Iteration better than Policy Iteration for first few iterations?

In Policy Iteration (PI), the action generated by the policy, whether it's optimal or not w.r.t the current value function $v(s)$. Whereas, in Value Iteration, the action is greedily generated w.r.t ...
1
vote
1answer
27 views

Can policy iteration use only the immediate reward for updates?

Is it still a policy iteration algorithm if the policy is updated optimizing a function of the immediate reward instead of the value function?
2
votes
1answer
41 views

Can I have different rewards for a single action based on which state it transitions to?

I am working on an MDP where there are four states and ten actions. I am supposed to derive the optimal policy to reach the desired state. At any state, a particular action can take you to any of the ...
1
vote
1answer
95 views

Unable to understand the second iteration update in value iteration algorithm for solving MDP

I am trying to understand the value iteration method for Markov Decision Process(MDP) and I was referring ot UC Berkeley's slides titled Markov Decision Processes and Exact Solution Methods On slide ...
3
votes
2answers
118 views

A few questions regarding the difference between policy iteration and value iteration

The question already has some answer. But I am still finding it quite unclear (also does $\pi(s)$ here mean $q(s,a)$ ?): The few things I do not understand are: Why the difference between 2 ...
1
vote
1answer
75 views

How do I apply the value iteration algorithm when there are two goal states?

I am working through the famous RL textbook by Sutton & Barto. Currently, I am on the value iteration chapter. To gain better understanding, I coded up a small example, inspired by this article. ...
2
votes
1answer
665 views

What is the time complexity of the value iteration algorithm?

Recently, I have come across the information (lecture 8 and 9 about MDPs of this UC Berkeley AI course) that the time complexity for each iteration of the value iteration algorithm is $\mathcal{O}(|S|^...
3
votes
1answer
878 views

Should the reward or the Q value be clipped for reinforcement learning

When extending reinforcement learning to the continuous states, continuous action case, we must use function approximators (linear or non-linear) to approximate the Q-value. It is well known that non-...
3
votes
1answer
64 views

In reinforcement learning, Is the optimal value (V*) corresponding to performing the best action in a given state?

it seems I am a little confused about the optimal value (V*) and optimal action-value (Q*) in reinforcement learning and just want some clarity because some blogs I read on Medium and GitHub are ...
5
votes
1answer
111 views

Reinforcement Learning (Fitted Q): Qn on Concept & Implementation

I hope to get some clarifications on Fitted Q-Learning ('FQL'). My Research So Far I've read Sutton's book (specifically, chp 6 to 10), Ernst et al and this paper. I know that ...