I've read in this: F. Rosenblatt, Principles of neurodynamics. perceptrons and the theory of brain mechanisms that in the Multilayer Perceptron the activation functions in the second, third, ..., are all non linear, and they all can be different. And in the first layer, they are all linear.
On what does this depends?
- When it is said "the neural network learns automatically", in colloquial words, what does it mean?
AFAIK, one first train the NN, then at some point NN learns. When does the "automatically" enters then?
Thanks in advance for your help.