Trying to make snake learn different directions

Question

I'm really new to neural networks. I'm trying to make a neural network with genetics algorithms which will make a snake learn to look for the food and avoid hitting his tail.

The thing is that I think that I've done it, but as there's no walls the snake learns to go one direction only without making a 180 turn [GIF HERE].

I've tried to incentivate mutations that make them turn by decreasing the score of the snakes that always takes the same directions, but it don't works. I've only made them dumber, needing more breeds to reach another "smart" linear snake.

I've made a network with 5 inputs:

• Food position relative to my position and direction (2 inputs. x and y)
• Nearest wall (my tail) if I turn left
• Nearest wall (my tail) if I do not turn
• Nearest wall (my tail) if I turn right

The output are 3, being the first one turning left, the second do not turn and the third going right. I make the snake go the highest one of the 3 outputs.

I've added 1 hidden layer of 8 neurons (inputs + outputs premise).

The way I calculate the score is:

• Each step, 1 point.
• Each food eaten, 10 points.
• If the snake lasts too much time without eating food, dies.
• If hits his tail, dies.

Then I save each time the direction this snake has gone (up, right, down, left) and increment them by one. When the snake dies, I weight the final score by the difference between the lowest and highest values. If the difference is high, they receive a big penalty (down to 0.25 of their score). This way if a snake is pretty much linear gets a high penalty and if a snake does a cool pattern, gets a low penalty.

Also, I keep a record of each time a direction change happens compared to the last direction change, so if a snake keeps going in circles don't gets a high score because of "cool pattern method" for going all 4 directions.

With all this, I don't understand why my best snakes are always the linear ones :-/

I spawn 20 snakes and get the best 4 of each generation when everyone dies.

For generations I use neataptics.js and for neural networks I use synaptics.js. I have a Fiddle here: http://jsfiddle.net/Llorx/gunsct5r/

At line 10 you can see the network definition. At 211 you can see the snake "view" (food position and walls. Where it gets the inputs) and at 164 you can see the score weight calculation depending on steps taken that I mentioned before.

All inputs are normalized from 0 to 1.

I'm sure that I'm doing, not one, but a lot of things pretty bad, as I'm a newbie on this, but some light on this will be really cool.

Answer 1

As far as I understand, you don't give the snake rewards for eating fast. In fact, "Each step, 1 point." means that the slower it finds food, the better. So why would it do turns, considering it is dangerous.

Answer 2

I observed the scoring system you used and compared it to the field size and starving rate. The worms either died or did not try enough hard with the prize of 10 of eating, so I added the amount to 1000.

This did still not introduce the 180 degrees turn yet. I had to adjust these settings on Neat:

  mutationRate: 0.7,
  mutationAmount: 6,

which introduced where the worms optimize their path as diagonal and when forced to change 90 direction, actually a tiny 180 happens on screen.

On same mutationRate and mutationAmount, but 10 points I once observed a horizontal 180, when the food appeared back and diagonal movement was not invented. Also I observed with this last setup a motion of constant 180s flow where consecutive vertical or horizontal 180s formed a two pixel thick line on screen.

These observations do not 100% do the same as your animation, but do instead introduce various ways that are quite near, or can be in certain conditions an initiative to such phenomenom.

White worm introduces two pixel line, red 90 degree turn with miniature 180.

EDIT

I changed the nice curve ratio like this:

     var ratio = 0.75 + ((mindir / maxdir) * 0.25* difstepsratio);

and around generation 200 there lived one worm to be able to move to all directions and make 180 if needed.

Unfortunately it died before I found screen shot buttons from my phone and (s)he happened to become second in the race and thus was left as a unicorn who existed only once.

Still remains mystery whether that mutation could survive and become the main species some day.

Further remarks:

I noticed that when mutation takes place in snake code the same pattern is replicated many times. If snake gets idea of consecutive turn lefts or turn rights when it is long, a suicide with tail is quite often taking place.

I tried to taggle this with a higher score on snake collision than starvation and around generation #180 the dna of suicidal 360 loop was born and remained some generations.

However, this generation did not evolve to the desired intelligent 180 worm I previously had seen. Instead, the suicide was so rewarding that it killed that branch quite soon away.