0
$\begingroup$

For a demo I need to develop an AI solution to learn how to play pong. I have the following requirements:

  • Computer needs to play against a human player. Learn while playing the game.
  • Poor AI result in the first game played.
  • A significant improvement must be noticeable within minutes.
  • Develop it within weeks. So open source.

Thus far I found during my search that I need to use the gym library as a platform. Pretty much all examples I've seen have used it. Google deepmind is by far the greatest expert in the field and I intend to use their work as it is open-source as well.

First question is: is there any other solution that meets my requirements and is open source?

I have some questions about the SimPLe package documentation, since it seems to be the latest produced by deepmind (Link). So anyone who has used it:

  • What do they mean with pretrained policy? Is this one fully capable to compete with a human? Or is it pretrained in the sense that CNN's can be pretrained: Only convolution layer is trained, they will still suck without more training?
  • Second chapter in the readme is called "Train your policy (model-free training)". Does this mean all the other options are not trained using a human? Using model-free training. From the paper it is clear that model-free training is inferior to World-model training, but if it is the only model I can use with humans training it, I will have to use it.
  • The third chapter describes World Model training (with random trajectories). It is not mentioned whether this uses human input for training or not. Does it? If not, how much time does it take to get a noticeable result (computer bouncing first ball)?
  • There is also an option to learn a Model-based training with pre-trained world models. How fast does this one learn? Does it already have a good performance from the start?
$\endgroup$
  • $\begingroup$ In my search engine for academic papers the "Simulated Policy Learning" algorithm (SimPLe) was described in 6 papers which were all published in 2019. To which of them the question targets directly? $\endgroup$ – Manuel Rodriguez Jul 5 at 17:24
  • $\begingroup$ Think it's this one arxiv.org/pdf/1903.00374.pdf $\endgroup$ – hasdrubal Jul 7 at 15:11
  • $\begingroup$ Excellent choice, the paper is about Atari gameplaying and was referenced by 14 articles so far. That means, the proposed "Simulated Policy Learning" (SimPLe) algorithm was analyzed and improved by other authors who have included in their academic writing a reference to the original one. The first task is to go carefully to the corpus of derived work to answer the open questions about pretrained world models and how to improve the accuracy with a human in the loop. $\endgroup$ – Manuel Rodriguez Jul 7 at 15:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.