A few months ago I made a simple game that is similar to the dinosaur game in Google Chrome - you jump over obstacles, or don't jump over levitating obstacles, and jump to collect bitcoins, which can be placed at 5 different heights. I used a very lightweight NN written by NYU professor Dan Shiffman, and within a few days the game and AI were done, starting off with a population of 200 jumpers, and a genetic algorithm (fitness function (points are given for avoiding obstacles and gathering bitcoins) and mutation), and it worked as it should.
However, this was only when the bitcoins and obstacles were not near each other, which I've been struggling with ever since.
So, I made a "training ground" where I put first a levitating obstacle, then a grounded one, and then a bitcoin after it, and then a bitcoin above a fourth grounded obstacle, and no matter how many times and how long I'd leave it to train, I'd always end up with identical behavior:
The first 3 obstacles are properly avoided, the first bitcoin is collected, and then jumpers would jump too early, land before the fourth "bitcoin" obstacle, and jump again, always crashing at almost the same place (across all generations, so even if I'd restart the training, they crash at the same place in the obstacle, with a deviation of a few pixels up or down). I added multilayer support to the NN, no improvements.
Today I replaced the NN with tensorflow.js, and I am getting identical behaviour.
My inputs are:
- distance to next obstacle
- altitude of next obstacle
- distance to next star
(for simplicity I removed the altitude of stars from the input, and keep them at a constant altitude)
I have 2 hidden layers (5 and 6 neurons), and 1 neuron in the output, which determines if the jumper should jump.
My only idea is that a neuron that decides when to jump because of the obstacle activates alongside the neuron that decides when to jump because of the bitcoin, their weights are summed up and a decision to jump too early is made.
I'll give somewhat of a (maybe bad) analogy:
If it takes you 1 month to prepare an exam, then, if you have 2 exams on the same day, you will start preparing them 2 months earlier. That logic works in this case, but not in my AI.
In the initial "toy neural network" I even added 8 layers of 12 neurons each, which I think is overkill for this case. In tf.js I used both sigmoid and relu activation functions. No matter what I did, no improvement.
Hope someone has an idea where I'm going wrong.