All Questions
3 questions
3
votes
2
answers
861
views
Apart from the state and state-action value functions, what are other examples of value functions used in RL?
In reinforcement learning, we often define two functions, the state-value function
$$V^\pi(s) = \mathbb{E}_{\pi} \left[\sum_{k=0}^{\infty}
\gamma^{k}R_{t+k+1} \Bigg| S_t=s \right]$$
and the state-...
- 41.4k
2
votes
1
answer
270
views
Are these two definitions of the state-action value function equivalent?
I have been reading the Sutton and Barto textbook and going through David Silvers UCL lecture videos on YouTube and have a question on the equivalence of two forms of the state-action value function ...
- 5,030
1
vote
1
answer
96
views
How are these two equations for the optimal state-value function equivalent?
By substituting the optimal policy $\pi_{\star}$ into the Bellman equation, we get the Bellman equation for $v_{\pi_{\star}}(s)=v_{\star}(s)$:
$$ v_{\star}(s) = \sum\limits_a \pi_{\star}(a|s) \sum\...
- 1,115