What is the impact of using multiple BMUs for self-organizing maps?

Question

Here's a sort of a conceptual question. I was implementing a SOM algorithm to better understand its variations and parameters. I got curious about one bit: the BMU (best matching unit == the neuron that is more similar to the vector being presented) is chosen as the neuron that has the smallest distance in feature space to the vector. Then I updated it and its neighbours.

This makes sense, but what if I used more than one BMU for updating the network? For example, suppose that the distance to one neuron is 0.03, but there is another neuron with distance 0.04. These are the two smallest distances. I would use the one with 0.03 as the BMU.

The question is, what would be the expected impacts on the algorithm if I used more than one BMU? For example, I could be selecting for update all neurons for which the distance is up to 5% more than the minimum distance.

I am not asking for code. I can implement it to see what happens. I am just curious to see if anyone have any insight on the pros and cons (except additional complexity) of this approach.

Answer 1

The following is my intuition about the behavior of the algorithm considering your question, based on my knowledge and experience with single BMU SOMs. I didn't verify it experimentally.

At early stages of training: It should disturb the SOM topology preserving properties, as you're assigning the same pattern to different (and probably distant) locations in the lattice, making the distribution multimodal. This is not terrible on its own, as different positions could encode different relationships to neighbors, overcoming the limitations of a single 2D/3D neighborhood. But if your SOM is not large enough, the multiple modes may collapse, resulting in unimodal distributions covering large portions of the map, wasting space. It all depends on your training schedule.

At later stages of training: If you started with a single BMU at early stages, it wouldn't provide any meaningful impact, as a SOM, by definition, clusters similar patterns at close positions of its lattice. In other words, the neurons closest to your input are already inside the BMU update radius. If you trained with multiple BMUs since the beginning, you would keep finding / activating the multiple maxima of your multimodal distributions for each pattern. The dimensionality reduction would become even more nonlinear than it already is with single BMUs. This may or may not be a problem, depending on your application.