I know the training of neural nets involves some sort of dimension manipulation to separate classes of different features.
If there is no variation of features, no matter for neural nets or simple dimension reduction methods (e.g. PCA, LDA) + clustering, neither of them are going to distinguish different classes.
In such sense, I would like to know the true power of neural nets:
How classification neural nets are different from simple dimension reduction + clustering?
or rephrase the question:
What value do neural nets add to solving classification problems in terms of its algorithmic architecture compared with simple dimension reduction + clustering?