I have a DQN model which has 3 features as inputs (properly normalized) and should output q-values for each of the 100 possible actions. However, prior to any training, I would like to examine the behavior of a random agent which acts by taking argmax over a randomly initialized Q-Network. However, the randomly initialized (with default pytorch initialization) Q-network always returns similar q-values for a wide variety of different inputs. Consequently, the random agent always chooses the same few actions (3 or 4) and never chooses any of the remaining (96-97) ones. During training, this doesn't change, as the agent keeps on using only a few actions (optimal policy consists in choosing a variety of different actions).

My DQN has 2 hidden layers each with 64 neurons and ReLU is used everywhere. I believe the problem is related to this one: https://stackoverflow.com/questions/57400555/keras-model-always-predicts-similar-results. I've also noticed that the random agent behaves "more randomly" when the number of actions is decreased to say 5 instead of 100.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.