# What is an appropriate fitness function for a simulated self-driving car?

I have been working for ages on a neuro-evolution AI program, where cars learn how to race around a track. Presently, I have a rudimentary fitness function that awards points for every degree traveled in the CW direction about the center of the window (removes points in CCW dir) and removes a certain amount of points for every collision that occurs. Cars also lose points for every moment they stay still and/or are colliding with something. My end goal is to create cars that can complete a track full of obstacles, faster than a human controlled car.

Is there a better fitness function that would result in more efficient cars that

1. make better use of their sensors,
2. are efficient in getting around the track (don't weave like a drunk driver), and
3. are faster in general (cars are too cautious, but this is a race!)

?

Half of the population seems to just spin around.

I have fully implemented my neuro-evolution program. Here's my implementation. However, you'll find that it isn't perfect.

How should I alter my fitness function to generate better driving cars?

Currently, the cars will only turn left or right if the outer sensors are activated. However, if the sensors directly in front are activated, the neural network is configured in such a way that "ignores" this signal. So, the cars crash head-on into obstacles but avoid obstacles in their periphery. I think this is due to the fact that the fitness function (which gives points for the displacement in 3 second time period) is too generous and removes the incentive for cars to avoid obstacles. I've tried altering the punishment for collisions and the reward for driving in the right direction but it still isn't performing the way I would like it to.

Here are some screenshots

• It would be nice to know how you exactly determine that the car travelled (rotated) 1 degree. Does the car get positive reward every time when it goes around its own little circle? – Molnár István Oct 7 '17 at 19:03
• It seems like you might want to add additional layering for score instead of just the immediate I moved one degree I get a point. Perhaps a moving window that calculates total degrees moved over the last TBD-seconds. I would probably give that more weight than near-term score. I would think that should help prevent weaving and circling. Which might beg the question of why keep a near-term score at all but I would keep the near-term score because I suspect it will help get around obstacles. – Dunk Oct 11 '17 at 17:21
• You also might need to take into account 'obstacles' movement. That includes speed and direction. Perhaps your cars are circling because they are trying to avoid other cars. However, in a real race, cars race side-by-side because they have to assume the other drivers are going to follow certain 'driving rules'. Thus, they don't need to be concerned about hitting cars driving next to them. Their biggest concern is not hitting the car in front of them. – Dunk Oct 11 '17 at 17:25

You can set a series of checkpoints - see this excellent article by Byte Master on codeproject.com:

https://www.codeproject.com/Articles/1220276/ReInventing-Neural-Networks

Implementing an autonomous car which contains only the environment plus a neural network is a very good opportunity to ignore the concept of a predictive physics engine. If the car only knows, what happens in the now, but has no idea what the next 10 seconds will bring, the car will fail any driver license test. Congratulations for implementing a near-sighted autonomous car which is even superior to comma.ai ... You can start the system, take away the arms from the wheel and the AI is driving in reverse direction like a ghost driver. I hope, that somewhere the red emergency button is there, otherwise the consequences may by fatal. Leaving out a predictive engine is a wonderful opportunity to not recognize potential alternatives. The car will recognize the obstacle only, if it is too late. That makes the system great for increasing the chaos and teach anti-patterns in human driving.

Bet let us take a detailed look into the fitness function. A fitness function drives the learning process of the neural network. It defines, which goal the system has, and which states should be avoided. Analyzing only the current time step, but not the future, helps to reduce the complexity, makes the programming easier and is the best advice for newbies. As a result the car isn't able to anticipate potential problems which are occurring in 10 seconds or in 20 seconds and is forced to crash with the obstacle because it will be too late to navigate around them. In your original posting you're talking about a 3 second time period which goes into the direction of a symbolic model. I would leave this dead end and concentrate only to the now, because this helps to improve my score.

You're correct that it is related to the fitness function, but only indirectly. Recall that a function is the mathematical embodiment of a conceptual relationship. The model is a motion model. Proximity is not simply applying the distance formula $$D = \sqrt{(x_2 - x_2)^2 + (y_2 - y_2)^2}$$.

Proximity as we "feel it" when driving a car is based on the risks we see in the trajectories of ourselves and the other moving objects along with the stationary objects. Mathematically, this is an integral over time of a distribution of probable locations.

Look for work in the aeronautics industry about collision avoidance. The mathematics and algorithms are well developed for antiaircraft weaponry (where the fitness function is the inverse of the one you want) and air traffic control, where the fitness is like yours except with an altitude dimension.