# Questions tagged [policies]

For questions related to policies (as defined in reinforcement learning or other AI sub-fields).

76 questions
Filter by
Sorted by
Tagged with
1 vote
506 views

### Why does the policy $\pi$ affect the Q value?

From my understanding, the policy $\pi$ is basically how the agent acts (i.e. the actions it will take in each state). However, I am confused about the Q value and how it is "affected" by a policy. ...
• 113
110 views

### Representation of state space, action space and reward system for Reinforcement Learning problem

I am trying to solve the problem of an agent dynamically discovering(start with no information about the environment) the environment and to explore as much of the environment as possible without ...
147 views

• 39.6k
136 views

### How do I compute the variance of the return of an evaluation policy using two behaviour policies?

Suppose there is an evaluation policy called $\pi_{e}$ and there are two behavior policies $\pi_{b1}$ and $\pi_{b2}$. I know that it is possible to estimate the return of policy $\pi_{e}$ through ...
• 481
1 vote
59 views

### Where does the expectation term in the derivative of the soft-max policy come from?

At slide 17 of the David Silver's series, the soft-max policy is defined as follows $$\pi_\theta(s, a) \propto e^{\phi(s, a)^T \theta}$$ that is, the probability of an action $a$ (in state $s$) is ...
1 vote
153 views

### Why does the value of state change depending on the policy used to get to that state?

From what I understand, the value function estimates how 'good' it is for an agent to be in a state, and a policy is a mapping of actions to state. If I have understood these concepts correctly, why ...
• 113
618 views

### Why is the derivative of this objective function 0 if the policy is deterministic?

In the Berkeley RL class CS294-112 Fa18 9/5/18, they mention the following gradient would be 0 if the policy is deterministic.  \nabla_{\theta} J(\theta)=E_{\tau \sim \pi_{\theta}(\tau)}\left[\left(\...
• 153
7k views

### What does "stationary" mean in the context of reinforcement learning?

I think I've seen the expressions "stationary data", "stationary dynamics" and "stationary policy", among others, in the context of reinforcement learning. What does it mean? I think stationary policy ...
• 428