I would like to work on a project where I teach an NN to play N64 games. To my current understanding, I would need an emulator?

I can do the Machine Learning side of it, im just unsure how I can give the NN access to the game's controls, such as left, right, up or down?

Where could I find more information on doing so and is using an Emulator the right path to take?

  • 1
    $\begingroup$ The Stanford university has published two papers in 2017 about the subject. In the same year which is 2017, the MIT has released also a paper about playing Super Smash Brothers with a neural network. What makes the system different from the earlier Mario AI challenge is, that 3d graphics are used in the game. $\endgroup$ – Manuel Rodriguez Oct 7 '19 at 22:31
  • $\begingroup$ Thank you very much. I’ll have a look at these papers and see what I can learn from them! $\endgroup$ – Eoin Mc Allister Oct 8 '19 at 7:24

The common options are:

  1. Implement a simple simulation of the game, perhaps without graphics. Train the agent on the simulation. This is usually not very satisfying, because the agent won't play the real game.
  2. Use an emulator that allows you to inspect the game's memory, and input values directly at the controls. BizHawk is a good choice for SNES and I think also for N64 emulation. Use a pipe to connect this program with your agent. Pass across the pipe any key values from the game's memory that represent the state of the game, and send back actions that represent the inputs the agent makes in that state. Usually this is the best option.
  3. Scrape the screen in real time. Tools like SkyScraper will allow you to capture the raw pixels displayed in a window. You should then be able to pass these pixels to your agents via a pipe. You may be able to use tools like Selenium to pass values back from the agent to the game. This route is usually a lot more intensive than option 2, but is the most realistic. It may also require very good hardware to keep up with the game in real time.
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.