My implementation of NEAT consistently fails to solve XOR completely. The species converge on different sub-optimal networks which map all input examples but one correctly (most commonly (1,1,0)). Do you have any ideas as to why that is?
Some information which might be relevant:
- I use a plain logistic activation function in each non-input node 1/(1 + exp(-x)).
- Some of the weights seem to grow quite large in magnitude after a large number of epochs.
- I use the sum squared error as the fitness function.
- Anything over 0.5 is considered a 1 (for comparing the output with the expected)
Here is one example of an evolved network. Node 0 is a bias node, the other red node is the output, the green are inputs and the blue "hidden". Disregard the labels on the connections.
EDIT: following the XOR suggestions on the NEAT users page of steepening the gain of the sigmoid function, a network that solved XOR was found for the first time after ca 50 epochs. But it still fails most of the time. Here is the network which successfully solved XOR: