All Questions
5 questions
1
vote
1
answer
96
views
How are these two equations for the optimal state-value function equivalent?
By substituting the optimal policy $\pi_{\star}$ into the Bellman equation, we get the Bellman equation for $v_{\pi_{\star}}(s)=v_{\star}(s)$:
$$ v_{\star}(s) = \sum\limits_a \pi_{\star}(a|s) \sum\...
- 1,115
3
votes
1
answer
2k
views
What is the difference between a greedy policy and an optimal policy?
I am struggling to understand what is the difference between an optimal policy and a greedy policy.
Let $F(r_{t+1},s_{t+1}| s_t,a_t)$ be the probability distribution accorting to which, given action $...
- 33
0
votes
1
answer
91
views
Are the state-action values and the state value function equivalent for a given policy?
Are the state-action values and the state value function equivalent for a given policy? I would assume so as the value function is defined as $V(s)=\sum_a \pi(a|s)Q_{\pi}(s,a)$. If we are operating a ...
2
votes
1
answer
269
views
Are these two definitions of the state-action value function equivalent?
I have been reading the Sutton and Barto textbook and going through David Silvers UCL lecture videos on YouTube and have a question on the equivalence of two forms of the state-action value function ...
- 5,030
3
votes
2
answers
861
views
Apart from the state and state-action value functions, what are other examples of value functions used in RL?
In reinforcement learning, we often define two functions, the state-value function
$$V^\pi(s) = \mathbb{E}_{\pi} \left[\sum_{k=0}^{\infty}
\gamma^{k}R_{t+k+1} \Bigg| S_t=s \right]$$
and the state-...
- 41.4k