How does dimensionality reduction occur in Self organizing Map (SOM)?

We have n dimension input for SOM and the output 2-D clusters. How does it happen?

SOM (Self-Organinizing Map) is a type of artificial neural network (ANN), introduced by the Finnish professor Teuvo Kohonen in the 1980s, that is trained using unsupervised learning to produce a low-dimensional, discretized representation of the input space of the training samples, called a map, and is therefore a method to do dimensionality reduction.

SOM produces a mapping from a multidimensional input space onto a lattice of clusters, i.e. neurons, in a way that preserves their topology, so that neighboring neurons respond to “similar” input patterns.

It uses three basic processes:

• Competition
• Cooperation

In competition, each neuron is assigned a weight vector with the same dimensionality d as the input space. Any given input pattern is compared to the weight vector of each neuron and the closest neuron is declared the winner.

In cooperation, the activation of the winning neuron is spread to neurons in its immediate neighborhood, and as a result this allows topologically close neurons to become sensitive to similar patterns. The size of the neighborhood is initially large, but shrinks over time, where an initially large neighborhood promotes a topology-preserving mapping and smaller neighborhoods allows neurons to specialize in the latter stages of training.

In adaptation, the winner neuron and its topological neighbors are adapted to make their weight vectors more similar to the input pattern that caused the activation.

So, a Self-Organizing Map (SOM) is to encode a large set of input vectors $$\textbf{x}$$ by finding a smaller set of “representatives” or “prototypes” or “code-book vectors” $$\textbf{w}$$ that provide a good approximation to the original input space. This is the basic idea of vector quantization theory, the motivation of which is dimensionality reduction or data compression. Performing a gradient descent style minimization on SOM's loss function (eg. the sum of the Euclidean distances between the input sample and each neuron) does lead to the SOM weight update algorithm, which confirms that it is generating the best possible discrete low dimensional approximation to the input space (at least assuming it does not get trapped in a local minimum of the error function).

To answer your question, you should take into consideration that the Dimensionality reduction takes place in fields that deal with large numbers of observations and/or large numbers of variables. Thus, SOM helps finding good "prototypes", in a way that each input pattern belongs to exactly one of them. As a result, the training instances are mapped to the training "prototypes" and the whole training set is mapped to a new one with less instances.

In addition, the "prototypes" neurons resulted by SOM can often be used as good centers in RBF networks or to classify patterns with the LVQ family algorithms.

• Good explanation about SOM but I could not find where you answered my ques Oct 4, 2020 at 15:24
• can you bold those lines Oct 4, 2020 at 15:25
• @rcvaram Thanks. It's fixed. Oct 4, 2020 at 15:34