8 votes

Deep Q-Learning "catastrophic drop" reasons?

This is a case of overfitting the Q function leading to compounding errors when selecting actions. You have been training your policy for too long on the same data distribution. Overfitting Q ...
devidduma's user avatar
  • 552
4 votes

Why does regular Q-learning (and DQN) overestimate the Q values?

The overestimation comes from the random initialisation of your Q-value estimates. Obviously these will not be perfect (if they were then we wouldn't need to learn the true Q-values!). In many value ...
David's user avatar
  • 4,910
3 votes

Q learning achieves small reward in simple dice game

In short, you do have a problem with your hyperparameters, but also single-step Q-Learning appears to struggle a lot with this simple looking environment. The environment seems simple because it is ...
Neil Slater's user avatar
  • 32.1k
1 vote
Accepted

Does "number of actions" refer to the number of actions taken or size of the action space?

The expression "number of actions" is being used in the same way in both cases. In fact, the letter $m$ is used in both cases. The number of actions (in the state $s$) is the number of ...
nbro's user avatar
  • 40.6k
1 vote

How to embed game grid state with walls as an input to neural network

If there can be multiple items on a square something more complex than one-hot is needed. As the name implies, only one category can be hot (set to true) in one-hot. You need multi-hot. Further, ...
foreverska's user avatar
  • 1,018
1 vote

How to embed game grid state with walls as an input to neural network

The way you describe with one hot encoding is correct. Note that how the state is encoded is a separate question from the neural network, so I'm not sure what convolutional neural networks have to do ...
Taw's user avatar
  • 1,251

Only top scored, non community-wiki answers of a minimum length are eligible