Skip to main content

Questions tagged [policies]

For questions related to policies (as defined in reinforcement learning or other AI sub-fields).

Filter by
Sorted by
Tagged with
8 votes
3 answers
17k views

What is the difference between a stochastic and a deterministic policy?

In reinforcement learning, there are the concepts of stochastic (or probabilistic) and deterministic policies. What is the difference between them?
nbro's user avatar
  • 40.9k
6 votes
1 answer
587 views

What is the relation between a policy which is the solution to a MDP and a policy like $\epsilon$-greedy?

In the context of reinforcement learning, a policy, $\pi$, is often defined as a function from the space of states, $\mathcal{S}$, to the space of actions, $\mathcal{A}$, that is, $\pi : \mathcal{S} \...
nbro's user avatar
  • 40.9k
16 votes
3 answers
4k views

Is the optimal policy always stochastic if the environment is also stochastic?

Is the optimal policy always stochastic (that is, a map from states to a probability distribution over actions) if the environment is also stochastic? Intuitively, if the environment is ...
nbro's user avatar
  • 40.9k
8 votes
1 answer
8k views

What is the difference between a stationary and a non-stationary policy?

In reinforcement learning, there are deterministic and non-deterministic (or stochastic) policies, but there are also stationary and non-stationary policies. What is the difference between a ...
nbro's user avatar
  • 40.9k
4 votes
1 answer
572 views

Why do we have two similar action selection strategies for UCB1?

In the literature, there are at least two action selection strategies associated with the UCB1's action selection strategy/policy. For example, in the paper Algorithms for the multi-armed bandit ...
nbro's user avatar
  • 40.9k
15 votes
4 answers
7k views

What does "stationary" mean in the context of reinforcement learning?

I think I've seen the expressions "stationary data", "stationary dynamics" and "stationary policy", among others, in the context of reinforcement learning. What does it mean? I think stationary policy ...
Paula Vega's user avatar
5 votes
2 answers
1k views

Given two optimal policies, is an affine combination of them also optimal?

If there are two different optimal policies $\pi_1, \pi_2$ in a reinforcement learning task, will the linear combination (or affine combination) of the two policies $\alpha \pi_1 + \beta \pi_2, \alpha ...
yang liu's user avatar
4 votes
1 answer
675 views

Can someone please help me validate my MDP?

Problem Statement : I have a system with four states - S1 through S4 where S1 is the beginning state and S4 is the end/terminal state. The next state is always better than the previous state i.e if ...
Bhavana's user avatar
  • 83
3 votes
1 answer
626 views

Are there some notions of distance between two policies?

I want to determine some distance between two policies $\pi_1 (a \mid s)$ and $\pi_2 (a \mid s)$, i.e. something like $\vert \vert \pi_1 (a \mid s) - \pi_2(a \mid s) \vert \vert$, where $\pi_i (a\mid ...
Felix P.'s user avatar
  • 297
2 votes
1 answer
371 views

Is the initialisation of $V(s)$ and $\pi(s)$ really arbitrary in policy iteration?

In Sutton and Barto's book (Reinforcement learning: An introduction. MIT press, 2018), the algorithm "Policy Iteration" is: Here, $V(s)$ is initialized arbitrarily, meaning that I can ...
Gregwar's user avatar
  • 121