Questions tagged [optimal-policy]
For questions related to the concept of "optimal policy" in reinforcement learning.
9
questions
2
votes
1answer
28 views
What does $v(S_{t+1})$ mean in the optimal state-action value function?
In Sutton & Barto's Reinforcement Learning: An Introduction page 63 the authors introduce the optimal state value function in the expression of the optimal action-value function as follows: $q_{*}(...
3
votes
1answer
39 views
How is $v_*(s) = \max_{\pi} v_\pi(s)$ also applicable in the case of stochastic policies?
I am reading Sutton & Bartos's Book "Introduction to reinforcement learning". In this book, the defined the optimal value function as:
$$v_*(s) = \max_{\pi} v_\pi(s),$$ for all $s \in \...
1
vote
1answer
36 views
Equivalence of the $Q(s,a)$ and $V(s)$ under optimality conditions?
Are the state-action values and the state value function equivalent for a given policy? I would assume so as the value function is defined as $V(s)=\sum_a \pi(a|s)Q_{\pi}(s,a)$. If we are operating a ...
5
votes
2answers
322 views
Given two optimal policies, is an affine combination of them also optimal?
If there are two different optimal policies $\pi_1, \pi_2$ in a reinforcement learning task, will the linear combination (or affine combination) of the two policies $\alpha \pi_1 + \beta \pi_2, \alpha ...
0
votes
0answers
21 views
Optimal path finding in multiagent game with collision avoidance
What algorithm to use when to find the optimal path from a starting point to the end point in a multiagent game with collision avoidance? I am in control of the agent while having zero ideas of where ...
4
votes
1answer
330 views
What's the optimal policy in the rock-paper-scissors game?
A deterministic policy in the rock-paper-scissors game can be easily exploited by the opponent - by doing just the right sequence of moves to defeat the agent. More often than not, I've heard that a ...
1
vote
2answers
95 views
Why is the optimal policy for an infinite horizon MDP deterministic?
Could someone please help me gain some intuition as to why the optimal policy for a Markov Decision Process in the infinite horizon case (agent acts forever) is deterministic?
2
votes
1answer
135 views
Equation not satisfied in Policy Iteration Algorithm
In equation 4.9 of Sutton and Barto's book on page 79, we have(for policy iteration algo):
$\pi ^{'}(s) = arg \max_{a}\sum_{s',r}p(s',r|s,a)[r+\gamma v_{\pi}(s')]$
where $\pi$ is the previous policy ...
4
votes
1answer
220 views
An example of a unique value function which is associated with multiple optimal policies
In the 4th paragraph of
http://www.incompleteideas.net/book/ebook/node37.html
it is mentioned:
Whereas the optimal value functions for states and state-action pairs are unique for a given MDP, ...
