The Kohonen network is one fully connected layer, which clusters the input into classes by a given metric. However, the one layer does not allow to operate with complex relations, that's why deep learning is usually used.
Is it possible then to make multi-layered Kohonen network?
AFAIK, the output of the first layer is already cluster flags, so the activation function on the non-last layers must be different from the original Kohonnen definition?
Kohonen networks by definition are single layer FCNN's, but what differentiates them from others is their unsupervised training procedure.
This procedure is a function of the input, the weights and some hyperparameters. This means if you have a multilayer network you could train only the final layers weights using this procedure. Think of it this way, let $f$ be the final layer, and $g$ be the composition of all other layers, such that the whole network $N(x) = f \circ g(x)$. Using kohonens training procedure, you could learn $f$'s weights where the input is $g(x)$ rather than x. But then how would you learn $g$'s weights?
So you could do this by iterative learning. let $N$ be a 2 layer network: $N = f_2 \circ f_1$. first learn $f_1$ using the single layer procedure, and now you could learn $f_2$ in the same manner, except the input features its trying to cluster would be $f_1(x)$ rather than $x$.
There does exist cons to this procedure though: Using a kohonen network at each layer will try to cluster it to the best of its abilities, not so that its final composition will be best (a common problem in many optimization procedures that arent end-to-end), but that its current representation is. This may lead to non-optimal results, not achieving that deep representation that you're looking for that you could achieve in autoencoders or other deeper unsupervised models.
