2
$\begingroup$

How can we train a competitive layer on non-normalized vectors using LVQ technique ?

an example is given below from Neural Network Design (2nd Edition) book

The net input expression for LVQ networks calculates the distance between the input and each weight vector directly, instead of using the inner product. The result is that the LVQ network does not require normalized input vectors. This technique can also be used to allow a competitive layer to classify non-normalized vectors. Such a network is shown in figure below.

enter image description here Use this technique to train a two-neuron competitive layer on the (non-normalized) vectors below, using a learning rate $\alpha=0.5$

$p_1=\begin{bmatrix} 1 \\ 1 \end{bmatrix}, p_2=\begin{bmatrix} -1 \\ 2 \end{bmatrix}, p_3=\begin{bmatrix} -2 \\ -2 \end{bmatrix}$

Present the vectors in the following order : $p_1, p_2, p_3, p_2, p_3, p_1$

Initial weights : $W_1=\begin{bmatrix} 0 \\ 1 \end{bmatrix}, W_2=\begin{bmatrix} 1 \\ o \end{bmatrix}$

$\endgroup$
0
$\begingroup$

We have 2 classes , 1 subclass for each class

\begin{equation} W^2=\begin{vmatrix} 1 & 0\\ 0 & 1\\ \end{vmatrix} \end{equation}

$p_1$:

\begin{equation} \alpha^1=compet(n^1)=compet\begin{vmatrix} ||w_1-p_1||\\ ||w_2-p_1||\\ \end{vmatrix} = compet\begin{vmatrix} ||\begin{vmatrix} 0 & 1\\ \end{vmatrix}^T-\begin{vmatrix} 1 & 1\\ \end{vmatrix}^T||\\ ||\begin{vmatrix} 1 & 0\\ \end{vmatrix}^T-\begin{vmatrix} 1 & 1\\ \end{vmatrix}^T||\\ \end{vmatrix}= compet(\begin{vmatrix} 1\\ 1\\ \end{vmatrix}) = \begin{vmatrix} 1\\ 0\\ \end{vmatrix}) \end{equation}

\begin{equation} \alpha^2=W^2\cdot\alpha^1= \begin{vmatrix} 1 & 0\\ 0 & 1\\ \end{vmatrix}\begin{vmatrix} 1\\ 0\\ \end{vmatrix}=\begin{vmatrix} 1\\ 0\\ \end{vmatrix} \end{equation}

\begin{equation} W_1(1) = W_1(0) + \alpha\cdot(p_1-W_1(0))=\begin{vmatrix} 0\\ 1\\ \end{vmatrix}+0.5\cdot(\begin{vmatrix} 1\\ 1\\ \end{vmatrix}-\begin{vmatrix} 0\\ 1\\ \end{vmatrix})=\begin{vmatrix} 0\\ 1\\ \end{vmatrix}+\begin{vmatrix} 0.5\\ 0\\ \end{vmatrix}=\begin{vmatrix} 0.5\\ 1\\ \end{vmatrix} \end{equation}

$p_2$ :

\begin{equation} \alpha^1=compet(n^1)=compet\begin{vmatrix} ||w_1-p_2||\\ ||w_2-p_2||\\ \end{vmatrix} = compet\begin{vmatrix} ||\begin{vmatrix} 0.5 & 1\\ \end{vmatrix}^T-\begin{vmatrix} 1 & 2\\ \end{vmatrix}^T||\\ ||\begin{vmatrix} 1 & 0\\ \end{vmatrix}^T-\begin{vmatrix} 1 & 2\\ \end{vmatrix}^T||\\ \end{vmatrix}= compet(\begin{vmatrix} 1.8027756377\\ 2.8284271247\\ \end{vmatrix}) = \begin{vmatrix} 0\\ 1\\ \end{vmatrix}) \end{equation}

\begin{equation} \alpha^2=W^2\cdot\alpha^1= \begin{vmatrix} 1 & 0\\ 0 & 1\\ \end{vmatrix}\begin{vmatrix} 0\\ 1\\ \end{vmatrix}=\begin{vmatrix} 0\\ 1\\ \end{vmatrix} \end{equation}

wrong class

\begin{equation} W_2(1) = W_2(0) - \alpha\cdot(p_2-W_2(0))=\begin{vmatrix} 1\\ 0\\ \end{vmatrix}-0.5\cdot(\begin{vmatrix} -2\\ 2\\ \end{vmatrix}=\begin{vmatrix} 2\\ -1\\ \end{vmatrix} \end{equation}

$p_3$ :

\begin{equation} \alpha^1= compet\begin{vmatrix} ||\begin{vmatrix} 0.5 & 1\\ \end{vmatrix}^T-\begin{vmatrix} -2 & 2\\ \end{vmatrix}^T||\\ ||\begin{vmatrix} 2 & -1\ \end{vmatrix}^T-\begin{vmatrix} -2 & 2\\ \end{vmatrix}^T||\\ \end{vmatrix}= compet(\begin{vmatrix} 70\\ 5\\ \end{vmatrix}) = \begin{vmatrix} 1\\ 0\\ \end{vmatrix}) \end{equation}

\begin{equation} \alpha^2=W^2\cdot\alpha^1\begin{vmatrix} 1\\ 0\\ \end{vmatrix} \end{equation}

wrong class

\begin{equation} W_1(2) = W_1(1) - \alpha\cdot(p_3-W_1(1))=\begin{vmatrix} 0.5\\ 0\\ \end{vmatrix}-0.5\cdot(\begin{vmatrix} -2.5\\ 1\\ \end{vmatrix}=\begin{vmatrix} 1.75\\ 0.5\\ \end{vmatrix} \end{equation}

$p_2$ :

\begin{equation} \alpha^1=compet(n^1)= compet\begin{vmatrix} ||\begin{vmatrix} 1.75 & 0.5\\ \end{vmatrix}^T-\begin{vmatrix} -1 & 2\\ \end{vmatrix}^T||\\ ||\begin{vmatrix} 2 & -1\ \end{vmatrix}^T-\begin{vmatrix} -1 & 2\\ \end{vmatrix}^T||\\ \end{vmatrix}= compet(\begin{vmatrix} 3.13\\ 4.24\\ \end{vmatrix}) = \begin{vmatrix} 1\\ 0\\ \end{vmatrix}) \end{equation}

\begin{equation} \alpha^2=W^2\cdot\alpha^1\begin{vmatrix} 1\\ 0\\ \end{vmatrix} \end{equation}

\begin{equation} W_1(3) = W_1(2) - \alpha\cdot(p_2-W_1(2))=\begin{vmatrix} 1.75\\ 0.5\\ \end{vmatrix}+0.5\cdot(\begin{vmatrix} -2.75\\ 1.5\\ \end{vmatrix}=\begin{vmatrix} 0.375\\ 1.25\\ \end{vmatrix} \end{equation}

$p_3$ :

\begin{equation} \alpha^1=compet(n^1)= compet\begin{vmatrix} ||\begin{vmatrix} 0.375 & 1.25\\ \end{vmatrix}^T-\begin{vmatrix} -2 & 2\\ \end{vmatrix}^T||\\ ||\begin{vmatrix} 2 & -1\ \end{vmatrix}^T-\begin{vmatrix} -2 & 2\\ \end{vmatrix}^T||\\ \end{vmatrix}= compet(\begin{vmatrix} 2.95\\ 5\\ \end{vmatrix}) = \begin{vmatrix} 1\\ 0\\ \end{vmatrix}) \end{equation}

\begin{equation} \alpha^2=W^2\cdot\alpha^1\begin{vmatrix} 1\\ 0\\ \end{vmatrix} \end{equation}

wrong class

\begin{equation} W_1(4) = W_1(1) - \alpha\cdot(p_3-W_1(3))=\begin{vmatrix} 0.375\\ 1.25\\ \end{vmatrix}-0.5\cdot(\begin{vmatrix} -2.375\\ 0.75\\ \end{vmatrix}=\begin{vmatrix} 1.5625\ 0.875\\ \end{vmatrix} \end{equation}

p_1 :

\begin{equation} \alpha^1=compet(n^1)= compet\begin{vmatrix} ||\begin{vmatrix} 1.5625 & 0.875\\ \end{vmatrix}^T-\begin{vmatrix} 1 & 1\\ \end{vmatrix}^T||\\ ||\begin{vmatrix} 2 & -1\ \end{vmatrix}^T-\begin{vmatrix} 1 & 1\\ \end{vmatrix}^T||\\ \end{vmatrix}= compet(\begin{vmatrix} 0.57\\ 2.23\\ \end{vmatrix}) = \begin{vmatrix} 1\\ 0\\ \end{vmatrix}) \end{equation}

\begin{equation} \alpha^2=W^2\cdot\alpha^1\begin{vmatrix} 1\\ 0\\ \end{vmatrix} \end{equation}

\begin{equation} W_1(5) = W_1(4) - \alpha\cdot(p_1-W_1(4))=\begin{vmatrix} 1.5625\\ 0.875\\ \end{vmatrix}+(\begin{vmatrix} -0.28125\\ 0.0625\\ \end{vmatrix}=\begin{vmatrix} 1.28125\ 0.9375\\ \end{vmatrix} \end{equation}

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.