# Questions tagged [policies]

For questions related to policies (as defined in reinforcement learning or other AI sub-fields).

76 questions
Filter by
Sorted by
Tagged with
58 views

### What happens when the probability of either one of the policies is 0 in Importance Sampling?

I have a general question about the methods that use importance sampling in RL. What happens when the probability of either one of the policies is 0?
1 vote
88 views

### What is the best distance measure between policies that are not probability distributions?

This question asks if there is a way to measure distance between policies that are in fact probability distributions. In the case of continuous control with deterministic policies where they take a ...
329 views

### Why is the policy not a part of the MDP definition?

I'm reading an article on reinforcement learning, and I don't understand why the agent's policy $\pi$ is not part of definition of Markov Decision process(MDP): Bu, Lucian, Robert Babu, and Bart De ...
14 views

### Finding an optimal action score function for Multi-Armed Bandit Problem

Considering a multi-armed bandit problem where there are : ...
236 views

### Is my understanding correct regarding the difference between policy and plan?

I am confused regarding the difference between policy and plan in reinforcement learning. According to my understanding, when we calculate the value of state using Bellman equation in deterministic ...
30 views

### Define possible?

In Reinforcement Learning, policies are defined in terms of possible actions (see for instance page 58 of the book by Sutton et al.). So, is any action that an agent has in its repertoire always "...
75 views

### Policy and Discount Factor

This question is similar to this question, however it has a different question. I'm learning MDP's and I'd like to know if I'm doing these exercises correctly: Consider the following MDP: Suppose a ...
1 vote
356 views

### If a policy is epsilon-greedy, is it technically stochastic?

Even though if exploration doesn't happen, it's deterministic.
234 views

16k views

### What is the difference between a stochastic and a deterministic policy?

In reinforcement learning, there are the concepts of stochastic (or probabilistic) and deterministic policies. What is the difference between them?
1 vote
135 views

### Is the policy gradient expression in Fundamentals of Deep Learning wrong?

I don't understand the policy gradient as explained in Chapter-9 (Deep Reinforcement Learning) of the book Fundamentals of deep learning. Here is the whole paragraph: Policy Learning via Policy ...
104 views

### Is a learned policy, for a deterministic problem, trained in a supervised process, a stochastic policy?

If I trained a neural network with 4 outputs (one for each action: move down, up, left, and right) to move an agent through a grid (deterministic problem). The output of the neural network is a ...
1 vote
651 views

### Why is the optimal policy for an infinite horizon MDP deterministic?

Could someone please help me gain some intuition as to why the optimal policy for a Markov Decision Process in the infinite horizon case (agent acts forever) is deterministic?
7k views

### What is the difference between a stationary and a non-stationary policy?

In reinforcement learning, there are deterministic and non-deterministic (or stochastic) policies, but there are also stationary and non-stationary policies. What is the difference between a ...
214 views

### How does the AlphaGo Zero policy decide what move to execute?

I was going through the AlphaGo Zero paper and I was trying to understand everything, but I just can't figure out this one formula:  \pi(a \mid s_0) = \frac{N(s_0, a)^{\frac{1}{\tau}}}{\sum_b N(s_0,...
106 views

I have implemented DQN algorithm and wonder why during testing, the best performance is achieved by a policy from about 300 episode, when mean Q values converge at about 800 episode? Mean Q-values ...
1k views

### What is a "learned policy" in Q-learning?

I am completing an assignment at the moment. One of the assignment questions asks how you identified the learned policy and how you obtained it. The question is a reinforcement learning question, and ...
7k views

### What does "stationary" mean in the context of reinforcement learning?

I think I've seen the expressions "stationary data", "stationary dynamics" and "stationary policy", among others, in the context of reinforcement learning. What does it mean? I think stationary policy ...
741 views

### Why does having a fixed policy change a Markov Decision Process to a Markov Reward Process?

If a policy is fixed, it is said that a Markov Decision Process (MDP) becomes a Markov Reward Process (MRP). Why is this so? Aren't the transitions and rewards still parameterized by the action and ...
54 views

103 views

### What kind of reinforcement learning method does AlphaGo Deepmind use to beat the best human Go player?

In reinforcement learning, there are model-based versus model-free methods. Within model-based ones, there are policy-based and value-based methods. AlphaGo Deepmind RL model has beaten the best Go ...
555 views

1 vote
153 views

### Why does the value of state change depending on the policy used to get to that state?

From what I understand, the value function estimates how 'good' it is for an agent to be in a state, and a policy is a mapping of actions to state. If I have understood these concepts correctly, why ...
53 views

### Off-policy full-random training in easy-to-explore environment

Let say we are in an environment where a random agent can easily explore all the states of an environment (for example: tic-tac-toe). In those environments, using off-policy algorithm, is it a good ...
449 views

### Why do we have two similar action selection strategies for UCB1?

In the literature, there are at least two action selection strategies associated with the UCB1's action selection strategy/policy. For example, in the paper Algorithms for the multi-armed bandit ...
1 vote
### Why doesn't value iteration use $\pi(a \mid s)$ while policy evaluation does?
I was looking at the Bellman equation, and I noticed a difference between the equations used in policy evaluation and value iteration. In policy evaluation, there was the presence of $\pi(a \mid s)$, ...