New answers tagged q-learning
2
If $\pi$ is a random policy, and after running through this algorithm, and for each state take the $\max Q(s,a)$ for all possible actions, why would that not be equal to $Q_{\pi^*}(s, a)$ (optimal Q function)?
Assuming that the estimates for $Q_{\pi}(s,a)$ have converged to close to correct values from many samples, then a policy based on $\pi'(s) = \text{...
0
It is quite common in DQN to instead of having the neural network represent function $f(s,a) = \hat{q}(s,a,\theta)$ directly, it actually represents $f(s)= [\hat{q}(s,1,\theta), \hat{q}(s,2,\theta), \hat{q}(s,3,\theta) . . . \hat{q}(s,N_a,\theta)]$ where $N_a$ is the maximum action, and the input the current state. That is what is going on here. It is ...
3
I think you are a bit confused about what is the update function and the target.
The equation you have there, and what is done in the video is the estimation of the true value of a certain state. In Temporal-Difference algorithms this is called the TD-Target.
The reason for your confusion might be that in the video he starts from the end state and goes ...
0
The popular Q-learning algorithm is known to overestimate action values under certain conditions. It was not previously known whether, in practice, such overestimations are common, whether they harm performance, and whether they can generally be prevented. In this paper, we answer all these questions affirmatively. In particular, we first show that the ...
Top 50 recent answers are included
