1
$\begingroup$

I was trying to code a single layer perceptron to understand binary AND:

1 1 1
0 1 0
1 0 0
0 0 0

I made up this code

#include <stdio.h>
#include <stdlib.h>
#include <math.h>
int main()
{
int input1, input2;
float weight1 = 0.3, weight2 = 0.4;
int output;
int training1, training2, expectedoutput;
int i;
int j=1;

//TRAINING
for(i=0; i<10000;i++)
{   


    if(j=1)
    {
        training1 = 0;
        training2 = 1;
        expectedoutput = 0;
    }
    if(j=2)
    {
        training1 = 1;
        training2 = 0;
        expectedoutput = 0;
    }
    if(j=3)
    {
        training1 = 0;
        training2 = 0;
        expectedoutput = 0;
    }
    if(j=4)
    {
        training1 = 1;
        training2 = 1;
        expectedoutput = 1;
        j=1;
    }
    output = weight1*training1 + weight2*training2 + 2;

    if(output != expectedoutput )
    {
        weight1 = weight1 + 0.156 * training1 * (expectedoutput - output);
        weight2 = weight2 + 0.156 * training2 * (expectedoutput - output);
    }
    j++;
}

printf("training done\n");
printf("weight1 = %f" "weight2 = %f\n",weight1,weight2);

//TESTING THE PERCEPTRON
for(i=0; i<5 ; i++)
{
scanf ("%d%d", &input1, &input2 );
output = weight1*input1 + weight2*input2;
printf("\n%d\n", output);
}
return 1;    
}

its supposed to input the 4 cases repeatedly and with a learning rate of 0.156 (which i set randomly) and i used the threshold as a weight of 2.

However after the training the perceptron still doesnt give the expected output. Is my understanding of perceptron rule wrong? Please help thank you!

$\endgroup$
  • $\begingroup$ For debugging the actual code, you probably want to try a different Stack such as Code Review or Cross Validated. The part of this question related to your understanding of the perceptron rule is definitely on topic here on the general AI forum. Welcome to AI! $\endgroup$ – DukeZhou Aug 27 '17 at 20:30
0
$\begingroup$

You have several flaws in your code that will lead to unexpected behavior. I identified the following flaws you need to address. Once you have fixed them, you should output your updated weights during each step of training to see what the learning algorithm actually does and if it is going in the right direction. I will not address pure style aspects (like using a simpler case-statement instead of a series of ifs and stuff like that) and focus on the real errors.

Skipping training case 1

Once you are in training case 4, you set j to 1, expecting to use training case 1 next. But later in your code you increase j by one (j++) and go directly to training case 2, skipping the first one. This means you only run through training case 1 during your first pass of the loop.

Assigning j a value instead of comparing it

if(j=1)

The if statement will always be true, because you do not compare j to 1 but you set j to 1. You basically assign a value and test if the value is true. Correct would be:

if(j==1)

Ignoring bias during test

During the test steps after training you forget to add the bias that you used during training:

output = weight1*input1 + weight2*input2

should actually be:

output = weight1*input1 + weight2*input2 + 2

Otherwise your perceptron behaves differently during training and testing.

Perceptron output must be 1 or 0

Those were all implementation issues. This point actually seems to come from a misunderstanding of perceptrons. A real perceptron can either fire or not, meaning it outputs either 1 or 0, nothing in between. You calculate your output the following way and use the output to compare to the expected output:

output = weight1*training1 + weight2*training2 + 2;

if(output != expectedoutput )

Your output here is a float value. Only in edge cases it will result in 1 or 0. What you actually want to do for a real perceptron is:

if (weight1*training1 + weight2*training2 + 2 >= 0) { output = 1 } else { output = 0 }

if(output != expectedoutput)

After you fixed those errors and studied the output of your learning algorithms, you should be able to get the perceptron to work. If you have any questions, please leave a comment and I will try to help.


Edit:

I don't have a C compiler here, so I quickly implemented your solution with my suggested fixes in Python 3.6 and the results are very close to the correct solution. The final weights obviously have to be -1 and -1 for an AND Gate with bias 2. The learning process gets very close but never reaches -1 and -1 exactly, which leads to the wrong output in the end. This is a very good showcase to illustrate why you prefer a sigmoid neuron instead of a perceptron in modern neural networks. Here is a relevant question from cross-validated concerning the difference of those two types of neurons.

$\endgroup$
  • 1
    $\begingroup$ I did the changes as suggested and indeed the learning process tends to 1. However i have a few questions. 1) why do the weights tend to -1 and -1? if i input 1 1, the output is -1*1 + -1*1 +2 =0. But the answer should be 1 according to AND truth table. It seems that the output is still not correct? 2) could someone explain or provide a link to the importance of learning rate here? i realize that the weights tend more to -1 if i use a small rate like 0.00015. and it gets less accurate when i use a higher learning rate. How do i determine the correct learning rate to use? Thank you! $\endgroup$ – Sheep Aug 28 '17 at 16:47
  • $\begingroup$ It tends towards 0 because 0 (or above) is the threshold for the perceptron to fire. Firing means "output 1" and not firing means "output 0". Using 0 is kind of arbitrary here, but that's what's usually used for perceptrons so I followed this convention. The learning rate just tells you how fast you are learning in each steps. A slower learning rate will require more training cycles but might get you closer to the "perfect" values in the end. There are mechanisms to determine a good constant or even dynamic learning rate, but in the beginning it is mostly trial and error. Hope that helps! $\endgroup$ – Demento Aug 28 '17 at 21:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.