Questions tagged [policies]
For questions related to policies (as defined in reinforcement learning or other AI sub-fields).
12
questions
8
votes
3
answers
17k
views
What is the difference between a stochastic and a deterministic policy?
In reinforcement learning, there are the concepts of stochastic (or probabilistic) and deterministic policies. What is the difference between them?
- 41.1k
6
votes
1
answer
602
views
What is the relation between a policy which is the solution to a MDP and a policy like $\epsilon$-greedy?
In the context of reinforcement learning, a policy, $\pi$, is often defined as a function from the space of states, $\mathcal{S}$, to the space of actions, $\mathcal{A}$, that is, $\pi : \mathcal{S} \...
- 41.1k
16
votes
3
answers
4k
views
Is the optimal policy always stochastic if the environment is also stochastic?
Is the optimal policy always stochastic (that is, a map from states to a probability distribution over actions) if the environment is also stochastic?
Intuitively, if the environment is ...
- 41.1k
8
votes
1
answer
9k
views
What is the difference between a stationary and a non-stationary policy?
In reinforcement learning, there are deterministic and non-deterministic (or stochastic) policies, but there are also stationary and non-stationary policies.
What is the difference between a ...
- 41.1k
4
votes
1
answer
610
views
Why do we have two similar action selection strategies for UCB1?
In the literature, there are at least two action selection strategies associated with the UCB1's action selection strategy/policy. For example, in the paper Algorithms for the multi-armed bandit ...
- 41.1k
15
votes
4
answers
7k
views
What does "stationary" mean in the context of reinforcement learning?
I think I've seen the expressions "stationary data", "stationary dynamics" and "stationary policy", among others, in the context of reinforcement learning. What does it mean? I think stationary policy ...
- 438
5
votes
2
answers
1k
views
Given two optimal policies, is an affine combination of them also optimal?
If there are two different optimal policies $\pi_1, \pi_2$ in a reinforcement learning task, will the linear combination (or affine combination) of the two policies $\alpha \pi_1 + \beta \pi_2, \alpha ...
- 53
4
votes
1
answer
694
views
Can someone please help me validate my MDP?
Problem Statement :
I have a system with four states - S1 through S4 where S1 is the beginning state and S4 is the end/terminal state. The next state is always better than the previous state i.e if ...
- 83
4
votes
3
answers
1k
views
Is the policy really invariant under affine transformations of the reward function?
In the context of a Markov decision process, this paper says
it is well-known that the optimal policy is invariant to positive affine transformation of the reward function
On the other hand, ...
- 181
3
votes
1
answer
667
views
Are there some notions of distance between two policies?
I want to determine some distance between two policies $\pi_1 (a \mid s)$ and $\pi_2 (a \mid s)$, i.e. something like $\vert \vert \pi_1 (a \mid s) - \pi_2(a \mid s) \vert \vert$, where $\pi_i (a\mid ...
- 297
2
votes
1
answer
388
views
Is the initialisation of $V(s)$ and $\pi(s)$ really arbitrary in policy iteration?
In Sutton and Barto's book (Reinforcement learning: An introduction. MIT press, 2018), the algorithm "Policy Iteration" is:
Here, $V(s)$ is initialized arbitrarily, meaning that I can ...
- 121
0
votes
1
answer
48
views
What is the best strategy to train a model with multi (sub)goals in the same environment?
To be able to explain my question I thought it is probably better to consider the following example: Let's take an environment, where a bridge crane need to lift a barrel from the position "start&...
- 204