Do I have to crossover my node genes in NEAT, and how?

Question

I'm currently trying to code the NEAT algorithm by myself, but I got stuck with two questions. Here they are:

What happens if during crossover a node is removed (or disabled) and there's a connection that was previously connected to that specific node? Because, in that case, some connections are no longer useful. Do I keep the useless connections or do I prevent this from happening? Or maybe I'm missing something?

Someone on AI SE said that:

You could:

1.) Use only the connection genes in crossover, and derive your node genes from the connection genes

2.) Test if every node is in use, and delete the ones that are not

But the problem with that is that my genomes will lose some complexity. Maybe I can use the nodes during crossover, and then disable the connections that were using this node. That way, I'm keeping the genotype complex, but the phenotype is still working.

Is there another way to workaround this problem or this is the best way?

Answer 1

Okay, I'm first going to review how NEAT works. I hope this helps you model NEAT successfully as a whole, not just limited to your question.

We use neuro-evolution to create a specific behavior that solves a given task. The behavior can be simple and complex.

Now let's focus on behavior... Different neural networks can create the same behavior (A.K.A the competing conventions problem).

We also want to end up with a neural network that is very efficient.

So we want to solve two problems with one algorithm (NEAT): find the behavior that solves a task and find the most efficient neural network that creates the behavior.

There's a simple way to search for the most efficient neural net: start with the simplest neural net and slowly build the neural net up.

The hard part is defining behavior (how do we define behavior in neural networks?). NEAT introduces their very interesting definition of behavior: an atomic unit of neural network behavior is the connection of the neural network: which node connects to which and with what weight.

Now I'm approaching your question(concern): you want to make sure your genomes can grow complex, which means that you want to preserve the capacity for your genome to express complex behavior. As the paper states, behavior does not have anything to do with nodes, let alone node genes. Behavior is a set of neural network connections.

So here's my answer to your question: care about crossing over connections, not for the node genes. Find an efficient way to calculate the node genes. For instance, you'll consume a lot of memory if you just concatenate node genes from two parents without overlaps. There is a chance that some of the node genes might not be in use (i.e. disabled connections, not inheriting some connection genes due to probability).

Hope this helps :)