I have a problem that arose as part of a NEAT (Neuro Evolution Through Augmenting Topologies) implementation that I am writing. I am wanting it to produce topologies or graphs that describe neural networks, similar to the one below.

Here, nodes 0 and 1 are inputs, and 4 is the output node, the rest of the nodes are hidden nodes. Each of these nodes can have some activation function defined for them (not necessary that all the hidden nodes have the same activation function)

Now, I want to perform the forward pass of this neural network with some data, and, based on how well it performed in that task, I assign it with a fitness value, which is used as part of the NEAT evolutionary algorithm to move towards better architectures and weights.

So, as part of the evolution process, I can have connections that can cause internal loops in the hidden layers and there is the possibility that a skip connection is made. Because of this, I feel the regular matrix-based forward pass (of fully connected MLPs) will not work in order to perform the forward pass of these evolved neural networks, and hence I want to know if an algorithm exists that can solve this problem.

In short, I want this neural network to just take the inputs and provide me outputs - no training involved at all, so I'm not interested in the back-propagation part now.

The only way to solve this that I see is to use something on the lines of a job queue (the queue will consist of the nodes that needs processing in order). I feel this is extremely inefficient and I cannot allocate this simulation method a proper stop condition. Or, even when to take output from the neural network graph and consider it.

Can anybody at least point me in the right direction?

  • $\begingroup$ The thing, I am extending the NEAT implementation. I have already implemented NEAT in two languages and those work. This is a interesting problem that I came upon when I considered evolving the neurons along with the network - I strongly think that it must have better convergence and should be much faster. The simple NEAT implementation is very straight forward - not much to think in there. Can I take my stack overflow reputation and bounty this question using that. $\endgroup$ Jan 4, 2021 at 17:51


You must log in to answer this question.