2
$\begingroup$

My implementation of NEAT consistently fails to solve XOR completely. The species converge on different sub-optimal networks which map all input examples but one correctly (most commonly (1,1,0)). Do you have any ideas as to why that is?

Some information which might be relevant:

  • I use a plain logistic activation function in each non-input node 1/(1 + exp(-x)).
  • Some of the weights seem to grow quite large in magnitude after a large number of epochs.
  • I use the sum squared error as the fitness function.
  • Anything over 0.5 is considered a 1 (for comparing the output with the expected)

Here is one example of an evolved network. Node 0 is a bias node, the other red node is the output, the green are inputs and the blue "hidden". Disregard the labels on the connections. enter image description here

EDIT: following the XOR suggestions on the NEAT users page of steepening the gain of the sigmoid function, a network that solved XOR was found for the first time after ca 50 epochs. But it still fails most of the time. Here is the network which successfully solved XOR: enter image description here

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

The problem was due to the following issues in my implementation:

  • The offspring generated in the crossover was not mutated (!)
  • The mutations did not occur with the expected frequencies (too few links and weight mutations)
  • The sigmoid activation had to be steepened

Another thing that previously caused issues was the network.activate function. Make sure that you wait for the network to stabilize when doing classification tasks, so all signals have time to propagate through the network.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .