Questions tagged [optimal-policy]
For questions related to the concept of "optimal policy" in reinforcement learning.
12
questions
0
votes
0
answers
35
views
Optimal decision with continuous, stochastic signals and rewards
I am performing a task, where I have to decide which projects to pursue at a given point in time, where the projects have different horizons of 30 minutes.
At a given point in time, forecasts are made ...
- 1
2
votes
1
answer
180
views
In which community does using a Bayesian regression model as a reward function with exploration vs. exploitation challenges fall under?
I am trying to find research papers addressing a problem that, in my opinion, deserves significant attention. However, I am having difficulty locating relevant information.
To illustrate the problem ...
- 35
0
votes
1
answer
108
views
How is policy iteration capable of improving on a deterministic policy?
Given a policy $\pi$ and the improved version upon it using policy iteration $\pi'$ we have, for $\forall s \in S$, $v_{\pi'}(s)\geq v_{\pi}(s)$.
I think the way we choose $\pi'$ makes it ...
- 573
0
votes
0
answers
172
views
Determine Gridworld values
I am learning Reinforcement learning for games following Gridworld examples. Apologies in advance if this is a basic question, very new to reinforcement learning.
I am slightly confused in scenarios ...
- 145
3
votes
1
answer
2k
views
What is the difference between a greedy policy and an optimal policy?
I am struggling to understand what is the difference between an optimal policy and a greedy policy.
Let $F(r_{t+1},s_{t+1}| s_t,a_t)$ be the probability distribution accorting to which, given action $...
- 33
3
votes
1
answer
339
views
Can an optimal policy have a value function that has a smaller value for a state than a non-optimal policy?
I'm starting to learn about the Bellman Equation and a question came to my mind.
A policy $\pi$ is optimal if the value $v_\pi(s)$ is greater or equal than the value $v_{\pi'}(s)$ for all states $s \...
- 133
2
votes
1
answer
76
views
What does $v(S_{t+1})$ mean in the optimal state-action value function?
In Sutton & Barto's Reinforcement Learning: An Introduction page 63 the authors introduce the optimal state value function in the expression of the optimal action-value function as follows: $q_{*}(...
- 573
3
votes
1
answer
110
views
How is $v_*(s) = \max_{\pi} v_\pi(s)$ also applicable in the case of stochastic policies?
I am reading Sutton & Bartos's Book "Introduction to reinforcement learning". In this book, the defined the optimal value function as:
$$v_*(s) = \max_{\pi} v_\pi(s),$$ for all $s \in \...
- 33
5
votes
2
answers
1k
views
Given two optimal policies, is an affine combination of them also optimal?
If there are two different optimal policies $\pi_1, \pi_2$ in a reinforcement learning task, will the linear combination (or affine combination) of the two policies $\alpha \pi_1 + \beta \pi_2, \alpha ...
- 53
5
votes
1
answer
833
views
What's the optimal policy in the rock-paper-scissors game?
A deterministic policy in the rock-paper-scissors game can be easily exploited by the opponent - by doing just the right sequence of moves to defeat the agent. More often than not, I've heard that a ...
- 1,131
1
vote
2
answers
646
views
Why is the optimal policy for an infinite horizon MDP deterministic?
Could someone please help me gain some intuition as to why the optimal policy for a Markov Decision Process in the infinite horizon case (agent acts forever) is deterministic?
- 1,131
4
votes
1
answer
329
views
An example of a unique value function which is associated with multiple optimal policies
In the 4th paragraph of
http://www.incompleteideas.net/book/ebook/node37.html
it is mentioned:
Whereas the optimal value functions for states and state-action pairs are unique for a given MDP, ...
- 143