1
$\begingroup$

I'm new to neural networks and I try to make a model that is guessing if a point is below or above relative to a function output. The idea is inspired from this video https://youtu.be/DGxIcDjPzac .

What am I doing wrong?

In the gif below I start the training but it seems that is not working. The blue line is the function (y = x + 50) and all the points above it should be green, but aren't. In order to simplify the example and to debug easier, I picked a simple function such that I can use only a perception for the model.

I also made a method backPropagationDebug(...) to display for the points that are predicted wrong all that matrices in each step, but I couldn't find what's wrong.

enter image description here

public void backPropagation(double[][] input, double[][] expected) {
    double[][][] outputs = getOutputs(input);

    double[][] currentOutput = outputs[outputs.length - 1];
    double[][] currentError = Matrix.subtract(expected, currentOutput);

    for (int i = brain.length - 1; i >= 0; i--) {
        final double[][] layer = brain[i];
        final double[][] previousOutput = outputs[i];

        final double[][] layerTranspose = Matrix.transpose(layer);
        final double[][] previousError = Matrix.multiply(layerTranspose, currentError);

        /* FIST BIT */
        double[][] errorSigmoid = Matrix.copyOf(currentError);

        for (int k = 0; k < errorSigmoid.length; k++) {
            errorSigmoid[k][0] *= - derivativeActivationFunction(currentOutput[k][0]);
        }

        /* SECOND BIT */
        final double[][] slopeMatrix = Matrix.multiply(errorSigmoid, Matrix.transpose(previousOutput));

        /* UPDATE THE WEIGHTS */
        for (int k = 0; k < layer.length; k++) {
            for (int l = 0; l < layer[0].length; l++) {
                layer[k][l] = layer[k][l] - learningRate * slopeMatrix[k][l];
            }
        }

        currentOutput = previousOutput;
        currentError = previousError;
    }
}

The backpropagation steps are inspired from this formulas: enter image description here enter image description here

(From: Make Your Own Neural Network By Tariq Rashid)

The code is on github: https://github.com/StamateValentin/Artificial-Intelligence-Playground/tree/7a7446b7faedd7673bc53a62304ff3a5180d77eb

The resources I used are in the README.md file.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Please include any mathematics using LaTeX; just pasting images of mathematics is frowned upon. $\endgroup$
    – The Pointer
    Jun 15, 2021 at 11:51
  • 1
    $\begingroup$ Welcome to the site. What is your specific question? Please put it somewhere it can be recognized easily. $\endgroup$
    – S2673
    Jun 15, 2021 at 17:18
  • 1
    $\begingroup$ I added the question. $\endgroup$
    – Valentin
    Jun 15, 2021 at 17:50
  • $\begingroup$ @Valentin, Thank you, that helps. Also, if you don’t put @[my username] at the beginning of your comment, then I won’t be notified that you responded. (More information about the comment system) $\endgroup$
    – S2673
    Jun 15, 2021 at 18:12
  • $\begingroup$ @S2673 Ok. thanks! $\endgroup$
    – Valentin
    Jun 15, 2021 at 18:20

0

Reset to default

You must log in to answer this question.