Where would I start if I wanted to create an AI agent to play a 2d game?

Question

I am keen on creating a little project that can play a fairly basic 2D game (more complex than say, snake but not as complex as mario kart) and would like some pointers on where to begin. I'm entirely new to any coding/programming but have a basic knowledge in machine learning. Is the first step to learn python, or unity? Then from there, what would I look at learning next? Any pointers are appreciated.

Answer 1

Theory

Learn the basic Reinforcement Learning (RL) concepts: Agent, Environment, Action, Reward, Return, Episode, Exploration. Maybe look at Q-Learning, but you don't need to dive deep yet.

Check out the Gymn environments or stable-baselines3. Both are Python libraries used for RL. Look at those for inspiration, not for getting programming experience. It's better to implement something much simpler on your own first.

Programming

If you're new to programming, even a single-player snakes will be a hard challenge. (But a good one, and there are several tutorials.) Interactive games are generally some of the hardest programming challenges.

I advice against Unity3D. Not because it is bad, but because it's too complex. The easiest way to start with programming is with text output. If you're doing a game played entirely by AI, you may even get away without GUI. (Just print the game-world as an array.)

Languages: Python is the natural choice because it's everywhere in machine learning and easy to pick up. C# if you want to use Unity3D later. Rust only if you are serious about investing time into programming. (Avoid C++ for now.) Python, Javascript and C# will all get you faster to something that works.

Step 1: Simple Environment

Some ideas:

• A racing-game where everything is a circle (walls, vehicles). Easy to detect collisions. Goal/Reward: going fast.
• A grid-world with random stuff lying around (obstacles, apples). Similar to snakes, but without the snake getting longer. Goal/Reward: apples collected.

Don't implement the AI yet. Instead, add an agent that ignores the inputs and makes a random choice every turn.

This will a) get you to something that runs, and b) a baseline score that your AI should beat. It's important to design the game such that random actions can sometimes score a point by chance. This gives the learning algorithm a signal to improve.

Step 2: Machine Learning

Now we need some kind of mapping between the agent inputs (e.g. the 8 neighbours in the grid, or distances to the next obstacle) to outputs (e.g. actions).

Personally I like to implement a neural network for that. It's actually simple (maybe 4 lines of code or so) if you only implement the forward-pass (no back-propagation) and choose the weights at random. With just random search, you should be able to discover weights that beat the baseline score.

Also, it's often better if the outputs are action probabilities (instead of just one action). A common way to do that with neural networks is a softmax layer.

Algorithms:

• Random search.
  • Really. It has been shown that some Atari games can be solved just by trying random weights on a neural network. Worth a shot.
  • "Random search" is also the name of a specific algorithm, which is not what I mean. The point is to first implement the simplest possible method and debug/simplify until it can make some progress. This builds your understanding how easy/difficult the task is to learn.
• Cross-Entropy Method (CEM). Fancy name, but can be simple to implement with basic probability theory (like, just a normal distribution).
  • Reference: Learning Tetris Using the Noisy Cross-Entropy Method
• Evolution strategies have been shown to be competitive with RL algorithms sometimes.
• A genetic algorithm. The Microbial GA is one that is easy to implement yourself.
• The neuroevolution algorithm NEAT is also popular for this kind of task.
• Any real RL algorithm from the stable-baselines3.

I wrote blogpost in 2019 about an experiment in the same spirit.

Answer 2

Basic knowledge in machine learning does typically not involve reinforcement learning, which is an ideal choice to make an agent in a video game. Reinforcement learning is famous for beating the simple Atari games, and later beating top players in more complicated games like StarCraft.

A good place to start would be pursuing skills and knowledge in both the field of game development and reinforcement learning. How about two independent courses at sites like Coursera and Udemy? There is probably a course that aims to solve some Atari games, which are avaliable at AI-Gym.

Then it will be trivial to determine if python or unity is right for you.

Answer 3

You can start with Monte Carlo Tree Search to play a 2D game. It is a pretty common algorithm in Reinforcement Learning. If you are comfortable enough with Neural Networks, you can use AlphaZero to create an algorithm to play a 2D game.

Here is a basic implementation of Monte Carlo Tree Search for TicTacToe: https://github.com/nityanandmathur/MCTS