Why would someone use NEAT over other machine learning algorithms?

Question

Why would someone use a neuroevolution algorithm, such as NEAT, over other machine learning algorithms? What situation would only apply to an algorithm such as NEAT, but no other machine learning algorithm?

Answer 1

The main difference leading to strengths and weaknesses of NEAT algorithm, is that it does not use any gradient calculations. That means for NEAT, neither the cost function, nor the activation functions of the neurons are required to be differentiable. In some circumstances - e.g. where agents directly compete, and you can select them for next generation if they win versus one or more opponents - you may not even need a cost function to optimise. Quite often the cost function can be a simple number generated from a complex evaluation of the network.

Therefore, NEAT can be used in situations where the formula for a cost function, in terms of single feed-forward runs of the network, is not very clear. It can also be used to explore activation functions such as step functions or stochastic firing neurons, where gradient based methods are difficult to impossible to apply.

NEAT can perform well for simple control scenarios, as a policy network that outputs actions given some sensor inputs. For example, it is popular to use it to create agents for racing games or simulated robotics controllers. When used as a policy-based controller, NEAT is in competition with Reinforcement Learning (RL), and has basic similarities with policy gradient methods - the nature of the controller is similar and often you could use the same reward/fitness function.

As NEAT is already an evolution-inspired algorithm, it also fits well with a-life code as the "brains" of simulated creatures.

The main disadvantage of NEAT is slow convergence to optimal results, especially in complex or challenging environments. Gradient methods can be much faster, and recent advances in Deep RL Policy Gradients (algorithms like A3C and DDPG) means that RL can tackle much more complex environments than NEAT.

I would suggest to use NEAT when:

• The problem is easy to assess - either via a measurement of performance or via competition between agents - but might be hard to specify as a loss function

• A relatively small neural network should be able to approximate the target function

• There is a goal to assess a non-differentiable activation function

If you are looking at a sequential control problem, and could use a standard feed-forward neural network to approximate a policy, it is difficult to say in advance whether NEAT or some form of Deep RL would be better.