4 votes

Deep Q-Learning "catastrophic drop" reasons?

This is a case of overfitting the Q function leading to compounding errors when selecting actions. You have been training your neural network as function approximator for too long on the same data ...
user avatar
  • 512
4 votes

Why does regular Q-learning (and DQN) overestimate the Q values?

The overestimation comes from the random initialisation of your Q-value estimates. Obviously these will not be perfect (if they were then we wouldn't need to learn the true Q-values!). In many value ...
user avatar
1 vote
Accepted

Does "number of actions" refer to the number of actions taken or size of the action space?

The expression "number of actions" is being used in the same way in both cases. In fact, the letter $m$ is used in both cases. The number of actions (in the state $s$) is the number of ...
user avatar
  • 33k

Only top scored, non community-wiki answers of a minimum length are eligible