# Neural Network trains towards 1 despite target

So I'm trying to make my first neural network and have just finished my back propagation functions. I got the algebra from brilliant and thought I'd understood it, but my bug proves otherwise. The bug itself is that after 500 iterations, both of my output neurons give a value of 1 despite my target values being 0 or 1. None of my training data gives both neurons outputting 1 at the same time, yet it happens. I'm sorry for posting a lot of code here, but I can't think of a better way of showing off the potential problem areas.

This calculates the error term for each neuron:

// Calculates error for each neuron in this layer, which we know isn't the output layer
// Error Term = derivation of sigmoid * sum of (weight to next layer * error term in next layer)
prevLayer.neurons.at(neuron).neuronData.error = prevLayer.neurons.at(neuron).CalculateSumWeightsError(this->neurons) *
prevLayer.neurons.at(neuron).GetDerivedValue(theta);


This is the contents of the loop that calculates the delta weights for each connection:

            // Calculates the connections new weight
// delta weight =
// -learning rate *
// Derivative of activation *
// input *
// sum of (weights * error term of next layer)

float newWeight = alpha * prevLayer.neurons.at(neuron).GetDerivedValue(theta);
newWeight *= prevLayer.neurons.at(neuron).neuronData.value;
newWeight *= this->neurons.at(errorNeuron).neuronData.sumErrorWeights;


This finds the total error and individual error terms in the output layer. It's part of a loop that iterates through all of them.

// Calculate individual error terms
// Error Terms = target - output
neuralLayers.back().neurons.at(output).neuronData.error = targets[output] - neuralLayers.back().neurons.at(output).GetActivatedValue(theta);
// Calculate Total Error for the output layer
// Total Error = sum of 1/2*(target - output)^2
errorTotal += 0.5f * FMath::Pow((targets[output] - neuralLayers.back().neurons.at(output).GetActivatedValue(theta)), 2.f);


Like I said, this is my first attempt at a neural network and thought I'd got it right this time round. Any help or resources would be very much appreciated as I genuinely don't know what's gone wrong.

• I don't have time to comb over the code, but when I was having gradient issues with my CNN, I found that doing a manual (ie, pen and paper) forward and backward pass of a simple architecture helped clean up any calculation issues quickly. You should try doing the same, take values at all layers and ensure they match what you manually calculated. Jan 12 at 4:47