0
$\begingroup$

I have a DQN model which has 3 features as inputs (properly normalized) and should output q-values for each of the 100 possible actions. However, prior to any training, I would like to examine the behavior of a random agent which acts by taking argmax over a randomly initialized Q-Network. However, the randomly initialized (with default pytorch initialization) Q-network always returns similar q-values for a wide variety of different inputs. Consequently, the random agent always chooses the same few actions (3 or 4) and never chooses any of the remaining (96-97) ones. During training, this doesn't change, as the agent keeps on using only a few actions (optimal policy consists in choosing a variety of different actions).

My DQN has 2 hidden layers each with 64 neurons and ReLU is used everywhere. I believe the problem is related to this one: https://stackoverflow.com/questions/57400555/keras-model-always-predicts-similar-results. I've also noticed that the random agent behaves "more randomly" when the number of actions is decreased to say 5 instead of 100.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.