5
$\begingroup$

I am new in the field of AI. I am working to create the flappy bird using Genetic Algorithm. After reading and seeing some examples, I saw that most implementations use a Neural Network + Genetic Algorithm and after certain generations, you achieve a very good agent that survives very long.

I currently struggling to implement the Neural Network since I have never taken a Machine Learning course. On many examples that I have read neural networks require training inputs and outputs. For the flappy bird, I can't think of output since you don't really know if the action of flapping will benefit you or not.

In the example that I followed, Synaptic.js is used and it is pretty straight-forward. However, in Python, I can't find a simple library that will initialize randomly and adjust the weights and biases depending on the good agents that survive longer.

What would be the right way to implement this Neural Network without having a training dataset?

Is there any way to create Flappy Bird without using Neural Networks, just Genetic Algorithm?

The example in Javascript that I am referring to: Flappy Bird Using Machine Learning

$\endgroup$
1
$\begingroup$

What you are trying to achieve, is a game that learns to play flappy bird. For doing this you need a neural network AND a genetic algorithm, those two things work together.
About your concerns on the output, you don't have to know if the action will benefit or not, i will soon explain why.

The neural network part

So, what you need is to know how to build a neural network, i don't know your knowledge about it, but i suggest starting from the basics. In this scenario, you need a feed forward neural network, because you just take the inputs from the current flappy bird scene/frame (such as the y position of the bird, the distance from the closes pipe ecc..) and feed it through a network that outputs either 1 or 0 (jump or don't jump) in the only output neuron we just decided it has.

In python you can implement a neural network from scratch, or using a neural network framework that does al the dirty work for you.

  • From scratch you would need to use numpy for matrix calculations, and you would need to learn matrix multiplication, dot products and all that fancy stuff (You can just let numpy taking care of the matrix calculations, but understanding how it works behind the scenes always helps understand new problems that you might come across when doing more advanced stuff)
  • Using a framework like Tensorflow for python, the only thing you need to do is find the right structure for the network you want to use. You will not have to worry about how activations work, or how the feed forward is performed (But again, it's a good thing to know when working with neural nets)

The genetic algorithm part or ""learning""

I say ""learning"" because at first sight it might look like learning, but really it is not. The genetic algorithm works like "the survival of the fittest", where the "smarter" birds, which are the ones that reached the higher score on the current generation, will have a chance to have their child little birds, that have the same brain as their parent, with either some minimal modifications, or a mix of their parent brains.
The process of this ""learning"", so the genetic algorithm, works like so:

  1. Create a generation of let's say 200 birds, every bird has a brain with random weights, so at the first run, they are all very...not smart
  2. The game starts, and every frame of the game, the brain of the bird recieves as input some data that is taken from the current frame ( y pos of the bird, distance from pipe...)
  3. The brain ( neural network ) of each bird, performs a feed forward with that data, and outputs what at the beginning is a very random result, let's say 0.75 for one bird
  4. At this point you decided that 0.75 is greater than 0.5, so you take that as a 1, which stands for "jump", while if it was 0.3, so 0, the bird does nothing and keeps falling
  5. Shortly the bird will die cause he has no idea of what he is doing, so he most likely collides with a pipe or the ground.
  6. After all birds met their fate, you see that some birds reached further than others, so you choose, for example, 5 of the best performing ones.
  7. Now you try to create a new generation of 200 birds using only the brains of those 5 that were choosen, by mixing and modifying theyr brains
  8. Now the new birds have a brand new brain, that in some cases might be better than the previous one, so chances are that some of those birds will reach a higher score, therefore flap further into the level.
  9. Repeat from point 6

In practice your "perform_genetic_algorithm" function in python, will have to choose the birds with the highest score, and as wild as it sounds, mix their brains and modify them, hoping that some modifications will improve the performance of the bird.

I can't think of output since you don't really know if the action of flapping will benefit you or not

The mechanism above explains why you basically do not care at all about the output, except saying to the game engine: "hey the bird decided to flap, do it". Whether it's the right action or not, doesn't matter, as the smarter birds are naturally gonna get further and so be more likely to be choosen for next generation.

Hopefully now it's all more clear.
Here is some useful links for building a neural network and for understanding the genetic algorithm:

  • How to build a neural network: I am linking this because it contains all useful information about how to build a very basic neural network in python. In your case, you would have to ignore all the part about backpropagation, loss & error calculation and SGD, and just look at the feed forward part.
  • How to build a neural network - 2: This is another example of building a neural network that i found really useful, probably it's simpler and more straight forward than the previous link, but again, the backpropagation part is not needed for this genetic based learning.
  • Video tutorials on genetic algorithm: This is a very long but very explanatory playlist of videos that dives into the nature of genetic algorithms and how to implement one
  • Genetic algorithm optimization: Other source about genetic algorithms
| improve this answer | |
$\endgroup$
0
$\begingroup$

I agree a neural net is a good start and you might want to add a constitutional neural network to list of models you want to test and evaluate.

However your question was really on getting something up and running using python. I don't have enough time for me to play games, much more let my computers do it for me... but I know someone who has done such a project.

Hopefully that helps.

| improve this answer | |
$\endgroup$
0
$\begingroup$

I would recommend learning about Reinforcement learning first. You don't need a dataset as you train your network by letting it play the game over and over again. but knowing how to do so doea mean finding out about markov decision process and how you can use the neural network to solve this.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.