5
$\begingroup$

As I know, a single layer neural network can only do linear operations, but multilayered ones can.

Alao I recently learned that finite matrices/tensors, which are used in many neural networks can only represent linear operations.

However multi-layered neural networks can represent nonlinear(even much more complex than being just a nonlinear!) operations.

What makes it happen? The activation layer?

$\endgroup$
  • $\begingroup$ Yes you are quite correct..The entire intuition of NN cannot be fit into a single answer so I suggest you take Andrew Ng's introductory course on machine learning in Coursera $\endgroup$ – DuttaA Aug 19 '18 at 16:54
  • $\begingroup$ The very quick answer is; yes, nonlinear activation functions in between linear transformations (matrix multiplications) allow for the "total" to represent non-linear functions. $\endgroup$ – Dennis Soemers Aug 19 '18 at 19:45
3
$\begingroup$

Nonlinear relations between input and output can be achieved by using a nonlinear activation function on the value of each neuron, before it's passed on to the neurons in the next layer.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.