I'm working on an implementation of NEAT, which evolves neural networks with small and sparse topologies.
Evaluating a sparse and possibly recurrent network requires a different approach than the matrix operations of dense networks, and I'm trying to wrap my head around the order in which nodes should be evaluated.
I've set up a simple example:
- Every node (except inputs) starts at 0. (t0)
- Every weight is 1. There's no activation function or biases.
Assuming that the algorithm has the same intuition we humans do: evaluate the nodes connected to the input, then the "aggregation" node, then the output. How should it decide whether to evaluate node  before node  or vice-versa?
By starting (t0) at , the value of  is 0. And vice-versa.
The side-effect of this behavior appears when comparing two networks. Say you have the network above, and a copy of it where  and  are inverted. I feel like both should return the same result for the same input in order for evolution to work properly.