# Backpropagation not working as expected

I'm new to neural networks and I try to make a model that is guessing if a point is below or above relative to a function output. The idea is inspired from this video https://youtu.be/DGxIcDjPzac .

What am I doing wrong?

In the gif below I start the training but it seems that is not working. The blue line is the function (y = x + 50) and all the points above it should be green, but aren't. In order to simplify the example and to debug easier, I picked a simple function such that I can use only a perception for the model.

I also made a method backPropagationDebug(...) to display for the points that are predicted wrong all that matrices in each step, but I couldn't find what's wrong.

public void backPropagation(double[][] input, double[][] expected) {
double[][][] outputs = getOutputs(input);

double[][] currentOutput = outputs[outputs.length - 1];
double[][] currentError = Matrix.subtract(expected, currentOutput);

for (int i = brain.length - 1; i >= 0; i--) {
final double[][] layer = brain[i];
final double[][] previousOutput = outputs[i];

final double[][] layerTranspose = Matrix.transpose(layer);
final double[][] previousError = Matrix.multiply(layerTranspose, currentError);

/* FIST BIT */
double[][] errorSigmoid = Matrix.copyOf(currentError);

for (int k = 0; k < errorSigmoid.length; k++) {
errorSigmoid[k][0] *= - derivativeActivationFunction(currentOutput[k][0]);
}

/* SECOND BIT */
final double[][] slopeMatrix = Matrix.multiply(errorSigmoid, Matrix.transpose(previousOutput));

/* UPDATE THE WEIGHTS */
for (int k = 0; k < layer.length; k++) {
for (int l = 0; l < layer[0].length; l++) {
layer[k][l] = layer[k][l] - learningRate * slopeMatrix[k][l];
}
}

currentOutput = previousOutput;
currentError = previousError;
}
}


The backpropagation steps are inspired from this formulas:

(From: Make Your Own Neural Network By Tariq Rashid)

The resources I used are in the README.md file.

• Please include any mathematics using LaTeX; just pasting images of mathematics is frowned upon. Jun 15, 2021 at 11:51
• Welcome to the site. What is your specific question? Please put it somewhere it can be recognized easily. Jun 15, 2021 at 17:18
• I added the question. Jun 15, 2021 at 17:50
• @Valentin, Thank you, that helps. Also, if you don’t put @[my username] at the beginning of your comment, then I won’t be notified that you responded. (More information about the comment system) Jun 15, 2021 at 18:12
• @S2673 Ok. thanks! Jun 15, 2021 at 18:20