1
$\begingroup$

In some Atari games in the Arcade Learning Environment (ALE), it is necessary to press FIRE once to start a game. Because it may be difficult for a Reinforcement Learning (RL) agent to learn this, they may often waste a lot of time executing actions that do nothing. Therefore, I get the impression that some people hardcode their agent to press that FIRE button once when necessary.

For example, in OpenAI's baselines repository, this is implemented using the FireResetEnv wrapper. Further down, in their wrap_deepmind (which applies that wrapper among others), it is implied that DeepMind tends to use this functionality in all of their publications. I have not been able to find a reference for this claim though.


My question is: is it common in published research (by DeepMind or others) to use the functionality described above? I'd say that, if this is the case, it should be explicitly mentioned in these papers (because it's important to know if hardcoded domain knowledge was added to a learning agent), but I have been unable to explicitly find this after looking through a wide variety of papers. So, based on this, I'd be inclined to believe the answer is "no". The main thing that confuses me then is the implication (without reference) in the OpenAI baselines repository that the answer would be "yes".

$\endgroup$
2
$\begingroup$

Based on this comment in the Issue I created about this question on github, it looks like there is confirmation that at least DeepMind does not use this kind of functionality in their Atari experiments, contrary to what is implied by the comments in the OpenAI baselines code.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.