# Train a competitive layer on nonnormalized vectors using LVQ technique

How can we train a competitive layer on non-normalized vectors using LVQ technique ?

an example is given below from Neural Network Design (2nd Edition) book

The net input expression for LVQ networks calculates the distance between the input and each weight vector directly, instead of using the inner product. The result is that the LVQ network does not require normalized input vectors. This technique can also be used to allow a competitive layer to classify non-normalized vectors. Such a network is shown in figure below.

Use this technique to train a two-neuron competitive layer on the (non-normalized) vectors below, using a learning rate $$\alpha=0.5$$

$$p_1=\begin{bmatrix} 1 \\ 1 \end{bmatrix}, p_2=\begin{bmatrix} -1 \\ 2 \end{bmatrix}, p_3=\begin{bmatrix} -2 \\ -2 \end{bmatrix}$$

Present the vectors in the following order : $$p_1, p_2, p_3, p_2, p_3, p_1$$

Initial weights : $$W_1=\begin{bmatrix} 0 \\ 1 \end{bmatrix}, W_2=\begin{bmatrix} 1 \\ o \end{bmatrix}$$

We have 2 classes , 1 subclass for each class

$$$$W^2=\begin{vmatrix} 1 & 0\\ 0 & 1\\ \end{vmatrix}$$$$

$$p_1$$:

$$$$\alpha^1=compet(n^1)=compet\begin{vmatrix} ||w_1-p_1||\\ ||w_2-p_1||\\ \end{vmatrix} = compet\begin{vmatrix} ||\begin{vmatrix} 0 & 1\\ \end{vmatrix}^T-\begin{vmatrix} 1 & 1\\ \end{vmatrix}^T||\\ ||\begin{vmatrix} 1 & 0\\ \end{vmatrix}^T-\begin{vmatrix} 1 & 1\\ \end{vmatrix}^T||\\ \end{vmatrix}= compet(\begin{vmatrix} 1\\ 1\\ \end{vmatrix}) = \begin{vmatrix} 1\\ 0\\ \end{vmatrix})$$$$

$$$$\alpha^2=W^2\cdot\alpha^1= \begin{vmatrix} 1 & 0\\ 0 & 1\\ \end{vmatrix}\begin{vmatrix} 1\\ 0\\ \end{vmatrix}=\begin{vmatrix} 1\\ 0\\ \end{vmatrix}$$$$

$$$$W_1(1) = W_1(0) + \alpha\cdot(p_1-W_1(0))=\begin{vmatrix} 0\\ 1\\ \end{vmatrix}+0.5\cdot(\begin{vmatrix} 1\\ 1\\ \end{vmatrix}-\begin{vmatrix} 0\\ 1\\ \end{vmatrix})=\begin{vmatrix} 0\\ 1\\ \end{vmatrix}+\begin{vmatrix} 0.5\\ 0\\ \end{vmatrix}=\begin{vmatrix} 0.5\\ 1\\ \end{vmatrix}$$$$

$$p_2$$ :

$$$$\alpha^1=compet(n^1)=compet\begin{vmatrix} ||w_1-p_2||\\ ||w_2-p_2||\\ \end{vmatrix} = compet\begin{vmatrix} ||\begin{vmatrix} 0.5 & 1\\ \end{vmatrix}^T-\begin{vmatrix} 1 & 2\\ \end{vmatrix}^T||\\ ||\begin{vmatrix} 1 & 0\\ \end{vmatrix}^T-\begin{vmatrix} 1 & 2\\ \end{vmatrix}^T||\\ \end{vmatrix}= compet(\begin{vmatrix} 1.8027756377\\ 2.8284271247\\ \end{vmatrix}) = \begin{vmatrix} 0\\ 1\\ \end{vmatrix})$$$$

$$$$\alpha^2=W^2\cdot\alpha^1= \begin{vmatrix} 1 & 0\\ 0 & 1\\ \end{vmatrix}\begin{vmatrix} 0\\ 1\\ \end{vmatrix}=\begin{vmatrix} 0\\ 1\\ \end{vmatrix}$$$$

wrong class

$$$$W_2(1) = W_2(0) - \alpha\cdot(p_2-W_2(0))=\begin{vmatrix} 1\\ 0\\ \end{vmatrix}-0.5\cdot(\begin{vmatrix} -2\\ 2\\ \end{vmatrix}=\begin{vmatrix} 2\\ -1\\ \end{vmatrix}$$$$

$$p_3$$ :

$$$$\alpha^1= compet\begin{vmatrix} ||\begin{vmatrix} 0.5 & 1\\ \end{vmatrix}^T-\begin{vmatrix} -2 & 2\\ \end{vmatrix}^T||\\ ||\begin{vmatrix} 2 & -1\ \end{vmatrix}^T-\begin{vmatrix} -2 & 2\\ \end{vmatrix}^T||\\ \end{vmatrix}= compet(\begin{vmatrix} 70\\ 5\\ \end{vmatrix}) = \begin{vmatrix} 1\\ 0\\ \end{vmatrix})$$$$

$$$$\alpha^2=W^2\cdot\alpha^1\begin{vmatrix} 1\\ 0\\ \end{vmatrix}$$$$

wrong class

$$$$W_1(2) = W_1(1) - \alpha\cdot(p_3-W_1(1))=\begin{vmatrix} 0.5\\ 0\\ \end{vmatrix}-0.5\cdot(\begin{vmatrix} -2.5\\ 1\\ \end{vmatrix}=\begin{vmatrix} 1.75\\ 0.5\\ \end{vmatrix}$$$$

$$p_2$$ :

$$$$\alpha^1=compet(n^1)= compet\begin{vmatrix} ||\begin{vmatrix} 1.75 & 0.5\\ \end{vmatrix}^T-\begin{vmatrix} -1 & 2\\ \end{vmatrix}^T||\\ ||\begin{vmatrix} 2 & -1\ \end{vmatrix}^T-\begin{vmatrix} -1 & 2\\ \end{vmatrix}^T||\\ \end{vmatrix}= compet(\begin{vmatrix} 3.13\\ 4.24\\ \end{vmatrix}) = \begin{vmatrix} 1\\ 0\\ \end{vmatrix})$$$$

$$$$\alpha^2=W^2\cdot\alpha^1\begin{vmatrix} 1\\ 0\\ \end{vmatrix}$$$$

$$$$W_1(3) = W_1(2) - \alpha\cdot(p_2-W_1(2))=\begin{vmatrix} 1.75\\ 0.5\\ \end{vmatrix}+0.5\cdot(\begin{vmatrix} -2.75\\ 1.5\\ \end{vmatrix}=\begin{vmatrix} 0.375\\ 1.25\\ \end{vmatrix}$$$$

$$p_3$$ :

$$$$\alpha^1=compet(n^1)= compet\begin{vmatrix} ||\begin{vmatrix} 0.375 & 1.25\\ \end{vmatrix}^T-\begin{vmatrix} -2 & 2\\ \end{vmatrix}^T||\\ ||\begin{vmatrix} 2 & -1\ \end{vmatrix}^T-\begin{vmatrix} -2 & 2\\ \end{vmatrix}^T||\\ \end{vmatrix}= compet(\begin{vmatrix} 2.95\\ 5\\ \end{vmatrix}) = \begin{vmatrix} 1\\ 0\\ \end{vmatrix})$$$$

$$$$\alpha^2=W^2\cdot\alpha^1\begin{vmatrix} 1\\ 0\\ \end{vmatrix}$$$$

wrong class

$$$$W_1(4) = W_1(1) - \alpha\cdot(p_3-W_1(3))=\begin{vmatrix} 0.375\\ 1.25\\ \end{vmatrix}-0.5\cdot(\begin{vmatrix} -2.375\\ 0.75\\ \end{vmatrix}=\begin{vmatrix} 1.5625\ 0.875\\ \end{vmatrix}$$$$

p_1 :

$$$$\alpha^1=compet(n^1)= compet\begin{vmatrix} ||\begin{vmatrix} 1.5625 & 0.875\\ \end{vmatrix}^T-\begin{vmatrix} 1 & 1\\ \end{vmatrix}^T||\\ ||\begin{vmatrix} 2 & -1\ \end{vmatrix}^T-\begin{vmatrix} 1 & 1\\ \end{vmatrix}^T||\\ \end{vmatrix}= compet(\begin{vmatrix} 0.57\\ 2.23\\ \end{vmatrix}) = \begin{vmatrix} 1\\ 0\\ \end{vmatrix})$$$$

$$$$\alpha^2=W^2\cdot\alpha^1\begin{vmatrix} 1\\ 0\\ \end{vmatrix}$$$$

$$$$W_1(5) = W_1(4) - \alpha\cdot(p_1-W_1(4))=\begin{vmatrix} 1.5625\\ 0.875\\ \end{vmatrix}+(\begin{vmatrix} -0.28125\\ 0.0625\\ \end{vmatrix}=\begin{vmatrix} 1.28125\ 0.9375\\ \end{vmatrix}$$$$