I am teaching bots to pick food on a playing field. Some food is poisonous and some is good.
- Poisonous food subtracts score points and good food adds.
- Food points vary based on its size.
- There is about 9:1 ratio of poisonous food to good food, so a lot more chances to end up in negative numbers.
- Food grows in points overtime.
- Food spoils after some predetermined size becoming poisonous.
The fitness function I use is simply counting points by the end of iterations. Bots might choose to eat it or skip it.
The problem I am having is that, in the first generation, most bots eat a lot of bad crap and the curious ones end up in negative numbers. So, mostly the ones that make it are the ones that are lazy and didn't eat or didn't head towards the food and most of the time the fittest for first few generations comes out with 0 points and 0 eats of any kind of food.
When trained for a long time, they just end up waiting for the food instead of eating multiple times. Often, while they wait, food goes bad and they just end up going to another food. This way, at the end of the iteration, I have some winners, but they are nowhere near the potential they could have been at.
I somehow need to weigh the importance of eating food. I want them to eventually learn to eat.
So I thought of this:
brain.score += foodValue * numTimesTheyAteSoFar
But this blows up the score too much and now the food quality is not respected and they just gulp on anything slightly above 0.