How do I design a fitness function that weighs the importance of eating food?

Question

Summary:

I am teaching bots to pick food on a playing field. Some food is poisonous and some is good.

Food Details:

• Poisonous food subtracts score points and good food adds.
• Food points vary based on its size.
• There is about 9:1 ratio of poisonous food to good food, so a lot more chances to end up in negative numbers.
• Food grows in points overtime.
• Food spoils after some predetermined size becoming poisonous.

Fitness Function:

The fitness function I use is simply counting points by the end of iterations. Bots might choose to eat it or skip it.

The Problem:

The problem I am having is that, in the first generation, most bots eat a lot of bad crap and the curious ones end up in negative numbers. So, mostly the ones that make it are the ones that are lazy and didn't eat or didn't head towards the food and most of the time the fittest for first few generations comes out with 0 points and 0 eats of any kind of food.

When trained for a long time, they just end up waiting for the food instead of eating multiple times. Often, while they wait, food goes bad and they just end up going to another food. This way, at the end of the iteration, I have some winners, but they are nowhere near the potential they could have been at.

Question:

I somehow need to weigh the importance of eating food. I want them to eventually learn to eat.

So I thought of this:

brain.score += foodValue * numTimesTheyAteSoFar

But this blows up the score too much and now the food quality is not respected and they just gulp on anything slightly above 0.

Answer 1

A human analogy can help you here (a variance).

Initialize all the agents with an initial value $x$; we will call this energyUnits. I Will talk later more about this.

Now, add some value, as an incentive, whenever the agent eats good food, to the energyUnits. You need to add a function that will keep decrementing the value of the agent's energyUnits, as humans degrade energy (calories) with time. We will call this function normalDegrade. This is the core part of the solution for your problem.

Now, for the bad (or poisonous) food you can be more creative with. You can simply subtract a given value whenever an agent eats poisonous food. Or you can extend your normalDegrade function with a very high downward slope. In this case, the energy units (value) of the agent will fall very rapidly. This will force the agent to look for good food to survive.

Since the ratio of food is 9:1 with poisonous, you need to initialize the value of $x$ (energyUnits) very high. You need to do some trial and error to find the right fit for you here.

Also, I am assuming that the agent is being removed from the population whenever the value of $x$ is zero or some negative value (which depends). This is important, as it makes sure that the algorithm is not wasting time in processing bad agents.

Because of this, another problem arises of the population coming to extinction. For this, you need to keep generating new agents for which any of the genetic algorithms will do. A new population with better parents of the already present generation will keep the population fit and efficient.

A good fitness function is a core to solving any problem of this kind, and sometimes it is hard to find. You might need to do some trial and error with different values to look for the right fit.