Questions tagged [optimal-policy]

For questions related to the concept of "optimal policy" in reinforcement learning.

Filter by
Sorted by
Tagged with
0 votes
0 answers
35 views

Optimal decision with continuous, stochastic signals and rewards

I am performing a task, where I have to decide which projects to pursue at a given point in time, where the projects have different horizons of 30 minutes. At a given point in time, forecasts are made ...
Mathman's user avatar
2 votes
1 answer
180 views

In which community does using a Bayesian regression model as a reward function with exploration vs. exploitation challenges fall under?

I am trying to find research papers addressing a problem that, in my opinion, deserves significant attention. However, I am having difficulty locating relevant information. To illustrate the problem ...
paul's user avatar
  • 35
0 votes
1 answer
108 views

How is policy iteration capable of improving on a deterministic policy?

Given a policy $\pi$ and the improved version upon it using policy iteration $\pi'$ we have, for $\forall s \in S$, $v_{\pi'}(s)\geq v_{\pi}(s)$. I think the way we choose $\pi'$ makes it ...
Daviiid's user avatar
  • 573
0 votes
0 answers
172 views

Determine Gridworld values

I am learning Reinforcement learning for games following Gridworld examples. Apologies in advance if this is a basic question, very new to reinforcement learning. I am slightly confused in scenarios ...
Krellex's user avatar
  • 145
3 votes
1 answer
2k views

What is the difference between a greedy policy and an optimal policy?

I am struggling to understand what is the difference between an optimal policy and a greedy policy. Let $F(r_{t+1},s_{t+1}| s_t,a_t)$ be the probability distribution accorting to which, given action $...
fennel's user avatar
  • 33
3 votes
1 answer
339 views

Can an optimal policy have a value function that has a smaller value for a state than a non-optimal policy?

I'm starting to learn about the Bellman Equation and a question came to my mind. A policy $\pi$ is optimal if the value $v_\pi(s)$ is greater or equal than the value $v_{\pi'}(s)$ for all states $s \...
raphael_mav's user avatar
2 votes
1 answer
76 views

What does $v(S_{t+1})$ mean in the optimal state-action value function?

In Sutton & Barto's Reinforcement Learning: An Introduction page 63 the authors introduce the optimal state value function in the expression of the optimal action-value function as follows: $q_{*}(...
Daviiid's user avatar
  • 573
3 votes
1 answer
110 views

How is $v_*(s) = \max_{\pi} v_\pi(s)$ also applicable in the case of stochastic policies?

I am reading Sutton & Bartos's Book "Introduction to reinforcement learning". In this book, the defined the optimal value function as: $$v_*(s) = \max_{\pi} v_\pi(s),$$ for all $s \in \...
Tamar's user avatar
  • 33
5 votes
2 answers
1k views

Given two optimal policies, is an affine combination of them also optimal?

If there are two different optimal policies $\pi_1, \pi_2$ in a reinforcement learning task, will the linear combination (or affine combination) of the two policies $\alpha \pi_1 + \beta \pi_2, \alpha ...
yang liu's user avatar
5 votes
1 answer
833 views

What's the optimal policy in the rock-paper-scissors game?

A deterministic policy in the rock-paper-scissors game can be easily exploited by the opponent - by doing just the right sequence of moves to defeat the agent. More often than not, I've heard that a ...
stoic-santiago's user avatar
1 vote
2 answers
646 views

Why is the optimal policy for an infinite horizon MDP deterministic?

Could someone please help me gain some intuition as to why the optimal policy for a Markov Decision Process in the infinite horizon case (agent acts forever) is deterministic?
stoic-santiago's user avatar
4 votes
1 answer
329 views

An example of a unique value function which is associated with multiple optimal policies

In the 4th paragraph of http://www.incompleteideas.net/book/ebook/node37.html it is mentioned: Whereas the optimal value functions for states and state-action pairs are unique for a given MDP, ...
Melanie A's user avatar
  • 143