3
$\begingroup$

I will be undertaking a project over the next year to create a self learning AI to play a racing game, currently the game will be Mario Kart 64.

I have a few questions which will hopefully help me get started:

  1. What aspects of AI would be most applicable to creating a self learning game AI for a racing game (Q-Learning, NEAT etc)
  2. Could a ANN or NEAT that has learned to play Mario Kart 64 be used to learn to play another racing game?
  3. What books/material should i read up on to undertake this project?
  4. What other considerations should i take throughout this project?

Thank you for your help!

$\endgroup$
  • $\begingroup$ Do you have access to a programmable environment for Mario Kart? Regardless of approach to AI/learning, you will need to have a system that can start the game, control it and receive feedback. $\endgroup$ – Neil Slater Oct 2 '18 at 13:09
  • $\begingroup$ I can read the game screen in using OpenCV, apply preprocessing to produce an input for the NN and then use a python module to simulate keypresses for the game, does that sound reasonable? $\endgroup$ – Ryan Oct 2 '18 at 13:14
  • $\begingroup$ If you use the game screen as input, I guess your model won't be able to play other game than Mario Kart (but I am not an expert so maybe I am wrong...) $\endgroup$ – Jérémy Blain Oct 2 '18 at 13:25
4
$\begingroup$

What aspects of AI would be most applicable to creating a self learning game AI for a racing game (Q-Learning, NEAT etc)

In general, you are looking at a problem that involves sequential decision making, in a machine learning context.

If you are wanting to build an agent that can learn by receiving screen images, then NEAT cannot scale to that complexity directly. Although there might be clever combinations of deep learning and evolutionary algorithms you could apply, the most heavily explored and likely successful solution will be found in Deep Reinforcement Learning. Algorithms like DQN, A3C, A2C, PPO . . . there are dozens to consider, but all are based around agents using samples of experience to update functions that measure either the "value" of acting in a certain way (a policy) or estimate the best policy directly.

Could a ANN or NEAT that has learned to play Mario Kart 64 be used to learn to play another racing game?

Within limits, yes. You will have built a system that takes pixel inputs, and outputs controller messages. If you re-start training from scratch on any compatible N64 game (with same screen resolution and same controller outputs) there is a chance it could learn to play that new game well. As other driving games on the N64 are a subset of all games, and more similar to each other than, say, a scrolling shooter or adventure game, then an agent that can successfully learn one will likely learn another too.

It is unlikely that a Mario Kart agent will immediately be good at another game without re-training. The visual and control differences will probably be too much. You could try though. An interesting experiment would be to take your trained agent, or some part of it, and see if starting with that improves learning time on a new game. This is called Transfer Learning.

What books/material should i read up on to undertake this project?

You will need at least the first parts of an introduction to Reinforcement Learning. If your goal is to aim to just make the agent, then you can skip theory-heavy parts, but within limits the more theory you understand, the easier it will be to change code features towards getting something working. I can suggest the following:

What other considerations should i take throughout this project?

Before you get started, you should know this could be an ambitious project, requiring a lot of compute time using currently-available toolkits. You need to think ahead a little, as you will be faced with these decisions:

  • Is it more important to you to make a working bot on this problem, or more important to understand the underlying theories so that you understand what is going on. You will need to do at least a little of both, but you can go strongly in either direction:
    • There are enough pre-built learning agents available on the internet that you might have a successful project by learning just enough to wire up a copy of the game, and then letting the learning agent do its job. Would this be enough for you, would you feel that you had solved your problem?
    • There is plenty of educational material about RL and how the maths behind it works. If you are interested in understanding that, and perhaps generating your own ideas for improvements to current algorithms, then you need to study that material harder. However, this may lead you away from your Mario Kart player goal, and you may never actually solve it, because far simpler problems that require less computing power are still actually interesting academically.

In addition, I can think of the following:

  • Building your Mario Kart player is a long-term goal. You will need to start with simpler agents in order to understand the techniques that you hope to apply to Mario Kart.

  • You may need more compute power to solve this game than you have available.

  • You will need to solve the issue of automating control of a N64 emulator. There is at least an existing emulator Mupen64Plus - I do not know whether it will be adequate for you, but at least one person has attempted to wrap this for automated learning, in the gym-mupen64plus project.

$\endgroup$
0
$\begingroup$

Before implementing an algorithm it is important to make clear what the single point of failure will be. That means a description, in which case the system will show a wrong behavior, because this is the point in which the project is over. Otherwise it will become an endless project which occupies many years of manpower. In case of neural networks game playing, the favourite failure point is an arror rate which becomes stable. That means, the weight changing algorithm has occupied all the processing power in full throttle but the error-rate of the NEAT neural network remains constant and doesn't improve anymore.

It is some kind of best practice method to develop the project purposeful into this direction. If you can capture a screenshot with a constant error rate but the agent on the screen doesn't drive very good your project is done. That means, the experiment failed, the limit of the neural network is reached and experts can think about what went wrong.

It is the same method like playing with domino bricks and if all pieces are falling down, the game is over. The question is only how to generate the failure state with less energy as possible. Some kind of trick is, if you draw the error chart by hand in Excel only to see how it looks, if a neural network has reached it's best policy and is not able to gain better results. But perhaps it is more useful, to really program a neural network with tensorflow or pybrain to get such a result. And don't be shy. I'm with you. My own NEAT project failed strongly. The curve with the constant error rate screenshot showed, that i didn't understood anything about neural networks.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.