How classification neural nets are different from simple dimension reduction + clustering?

Question

I know the training of neural nets involves some sort of dimension manipulation to separate classes of different features.

If there is no variation of features, no matter for neural nets or simple dimension reduction methods (e.g. PCA, LDA) + clustering, neither of them are going to distinguish different classes.

In such sense, I would like to know the true power of neural nets:

How classification neural nets are different from simple dimension reduction + clustering?

or rephrase the question:

What value do neural nets add to solving classification problems in terms of its algorithmic architecture compared with simple dimension reduction + clustering?

Answer 1

PCA is linear, NN are nonlinear, more generally it's a universal function approximator.

That said basic NN are not terribly usefull, the real value of NN is for structured data which is obvious for us but hard to describe analytically. Basically NN architecture has been designed to learn this structure using operations like convolution and maxpooling.

There are still problems of course, and in some ways I think clustering algorithms still have value for things like anomaly detection.