# Understanding the perceptron algorithm in the book “A Course in Machine Learning”

The following text is from Hal Daumé III's "A Course in Machine Learning" online text book (Page-41).

I understand that $$D$$ is the size of the input vector $$x$$.

1. What is $$y$$? Why is it introduced in the algorithm? How/where/when is the initial value of $$y$$ given?

2. What is the rationale of testing $$ya \leq 0$$ for updating weights?

Y is the desired output of the perceptron (often referred to as target) , for the given set of input vectors.

Rationale behind Y.a<=0 :

Prerequisite knowledge :

• A=A-B : Moves vector A away from direction of vector B

• A=A+B : Moves A in the direction of B

• A (.) B >0 ; A vector is directed acutely (<90 deg.) towards B vector

• A (.) B <0 ; A vector is directed away from (>90 deg.) from B vector [(.) denotes dot (scalar) product and bold letters indicate vectors]

• W is augmented vector (includes threshold as another weight along with normal input weights)

• X is augmented input (includes -1 as extra input (corresponding to threshold) along with other normal inputs

• a(activation) = W (.) X

• a >=0 ; Perceptron output 1

• a<0 ; Perceptron output -1 (Not zero as implicit in the given algorithm I think)

Now Rationale :

(Y.a<0)

This means Either of the following :

• Y=-1 and a>0 ; in this case the target output is -1 but as a>0 so the Perceptron outputs 1. So we must move the weight vector away from this set of input vector, so that angle between them increases and the dot product (a) becomes < 0 so that we can get the target output. Hence : W=W-X

Or ,W=W + (-1)*X Or, W=W+YX

• Y=1 and a<0

This means the target output is 1 but as the activation is <0 so Perceptron is outputting -1. So we must move the weight vector close to this set of input vector so that the activation can become >0 (angle decreases) and the Perceptron can output the desired output. So : W = W+X Or W=W + YX

Again , Y.a=0 is a boundary case. By now I think you can understand the rationale behind Y.a=0. If any doubt , comment to this answer , I will explain it.

Sorry for so much long answer though. :) :)

• Is that $W(.)A = = W^T.A$? – user3642 Dec 26 '16 at 7:47
• What are A and T in the above comment? – Programmer Dec 26 '16 at 8:15
• Sorry for that, W(.)X == W^T.X. I.e. dot product of transpose of W with X. – user3642 Dec 26 '16 at 8:51
• No they won't be same until you perform transpose on both the vectors. [ Although I assume that by transpose you mean reversing the components of the vectors. I.e , xi+yj+zk -> zi+yj+xk ] as transpose is inherent concept with matrix not vectors. – Programmer Dec 26 '16 at 9:43