# Is a multi-layer Kohonen network possible?

The Kohonen network is one fully connected layer, which clusters the input into classes by a given metric. However, the one layer does not allow to operate with complex relations, that's why deep learning is usually used.

Is it possible then to make multi-layered Kohonen network?

AFAIK, the output of the first layer is already cluster flags, so the activation function on the non-last layers must be different from the original Kohonnen definition?

This procedure is a function of the input, the weights and some hyperparameters. This means if you have a multilayer network you could train only the final layers weights using this procedure. Think of it this way, let $$f$$ be the final layer, and $$g$$ be the composition of all other layers, such that the whole network $$N(x) = f \circ g(x)$$. Using kohonens training procedure, you could learn $$f$$'s weights where the input is $$g(x)$$ rather than x. But then how would you learn $$g$$'s weights?
So you could do this by iterative learning. let $$N$$ be a 2 layer network: $$N = f_2 \circ f_1$$. first learn $$f_1$$ using the single layer procedure, and now you could learn $$f_2$$ in the same manner, except the input features its trying to cluster would be $$f_1(x)$$ rather than $$x$$.
• What did you mean by learn $f_1$. Afaik, in ideal cases kohonen is supposed to output a spike in a single output node, so of we learn this and put another layer on top, I don't see any kind of advantage, unless we do incomplete learning where the spike is smooth and more of a hill, but then again I am unsure how that'll provide any extra help. – user9947 Jul 17 '19 at 19:08