New answers tagged


Rosenblatt was probably discussing a specific architecture, for which there are many. However, for general purpose feed-forward back-propagation ANNs used for function aproximation and classification analysis, you can use whatever activation functions you want on the input-side, hidden layers, and output-side. Examples are identity, logistic, tanh, ...


For the multiclass SVM, there will be an ensembling effect since you are learning 5*4=20 1vs1 classifiers. It could be an interesting experiment to try the same thing with simple neural networks. Also, since you are standardizing the inputs you could try tanh activations on the first layer after the input. I presume you are using softmax on the output layer.


According to wikipedia of backpropagation: In fitting a neural network, backpropagation computes the gradient of the loss function during supervised learning with respect to the weights of the network for a single input–output example, and does so efficiently, unlike a naive direct computation of the gradient with respect to each weight individually. ...

Top 50 recent answers are included