9
$\begingroup$

DeepMind state that their deep Q-network (DQN) was able to continually adapt its behavior while learning to play 49 Atari games.

After learning all games with the same neural net, was the agent able to play them all at 'superhuman' levels simultaneously (whenever it was randomly presented with one of the games) or could it only be good at one game at a time because switching required a re-learn?

$\endgroup$
  • $\begingroup$ "After learning all games with the same neural net". Does this mean same NN architecture OR same architecture and single set of weights ? $\endgroup$ – Ankur Oct 20 '16 at 6:13
  • $\begingroup$ @Ankur actually I'm not sure - it's my (limited) understanding that they used the same architecture and didn't reset the weights between games. $\endgroup$ – Dion Oct 20 '16 at 8:05
2
$\begingroup$

Switching required a re-learn.

Also, note that:

We use the same network architecture, learning algorithm and hyperparameters settings across all seven games, showing that our approach is robust enough to work on a variety of games without incorporating game-specific information. While we evaluated our agents on the real and unmodified games, we made one change to the reward structure of the games during training only.

and

the network has outperformed all previous RL algorithms on six of the seven games we have attempted and surpassed an expert human player on three of them.

$\endgroup$
1
$\begingroup$

Switching requires relearning, the network did not have a single set of weights that allowed it to play all games well. This is due to the catastrophic forgetting problem.

However, recent work has been done to overcome this problem:

"Overcoming catastrophic forgetting in neural networks", 2016

Paper: https://arxiv.org/pdf/1612.00796v1.pdf

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.