New answers tagged value-functions
4
The core differences between using $V(s)$ or $Q(s,a)$ are:
$V(s)$ cannot be used stand-alone to decide a policy.
You either need a separate policy function $\pi(a|s)$ that it is the value function for, or you can derive a policy from it if you have access to the environment's distribution model $p(r,s'|s,a)$ - the probability of receiving reward $r$ and ...
1
Can't it be that the optimal policy thinks a state isn't that good and gives him a low value but perform best in comparison with other policies which have higher values for this state?
No, this is not possible, and this is part of the definition of an optimal policy.
You are asking if it is possible to construct a policy $\pi^?$ where for some state $s_z$, $...
Top 50 recent answers are included