Can NEAT produce output which has no connection with any other node?

Question

I recently developed a little tool called Neat Cars where you can drawa track, place a car and watch the NEAT algorithm in action (including the ANN).

However, I saw something that caught my attention:

Of course, the first thing I did why asking ChatGPT after seeing this as I don't think the drawing is lying. It gave me this exact explaination:

In NEAT (NeuroEvolution of Augmenting Topologies), it is possible for an output node to have no connections with any other node in the neural network but still produce an output. This can happen when the output node is initially connected to an input node or when it is connected to a hidden node that later becomes disconnected from the network.

When a neural network is evolving through NEAT, the algorithm may add or remove connections or nodes during the optimization process. If a connection between the output node and other nodes is removed, the output node can still produce an output because it may have learned to do so during previous generations or training iterations.

Additionally, in NEAT, some output nodes may be designated as "structural" nodes, which means they are always included in the network regardless of their connectivity. These structural nodes can also produce outputs even if they are not connected to any other nodes.

In summary, while it is not common in traditional neural network architectures for an output node to produce output without any input connections, it is possible in NEAT due to the way the algorithm evolves and optimizes neural networks.

But still, I don't think it works that way. I'm still having doubts.

Answer 1

To answer the main question:

Can NEAT produce output which has no connection with any other node?

Yes. This is a common property of most artificial neurons, and not really to do with NEAT.

If you have a set of inputs to an artificial neuron $x_i$, weighted by weights $w_i$, and the output is $y$, then one way to write the function for a single neuron is:

$$y = f(b + \sum_{\forall i}w_i x_i)$$

where $f$ is the activation function (in your case you have said this is $\text{tanh}$) and $b$ is the neuron's bias. The many other ways of writing this are usually to allow for looking at the larger structure of the network, and are equivalent when you consider just one neuron.

Your disconnected neurons have an empty set of inputs, but that is not a problem for typical NN code used in NEAT. It means the NN's function in your case is $y = \text{tanh}(b)$, which will be the same value no matter what the rest of the network does.

As your action decision is an $\text{argmax}$ over all outputs, and the other outputs can vary, sometimes they will be lower than the fixed value, and in fact the neural network does not lose any major ability by having such a default for one of its outputs. NEAT networks do tend towards simplicity and can evolve to drop connections that are non-necessary.

However, if two outputs were disconnected then one action would never be selected, which would result in poor performance in this problem (although you could potentially have all turns in the same direction for some tracks, or maybe no need to brake).

Of course, the first thing I did why asking ChatGPT

I recommend that you do not do this for anything important. ChatGPT when asked to explain technical things will output "explainy" text with some facts, some errors and a word salad of ideas that roughly link to your technical subject. Its answer to you that you quoted is basically nonsense with a few facts thrown in that are not explanatory for your question. Maybe a future version will be better at explaining technical subjects, but this version is not good for that use, and you will waste time trying to understand nonsense.