6
$\begingroup$

As I know, a single layer neural network can only do linear operations, but multilayered ones can.

Also, I recently learned that finite matrices/tensors, which are used in many neural networks, can only represent linear operations.

However, multi-layered neural networks can represent non-linear (even much more complex than being just a nonlinear) operations.

What makes it happen? The activation layer?

$\endgroup$
  • 1
    $\begingroup$ The very quick answer is; yes, nonlinear activation functions in between linear transformations (matrix multiplications) allow for the "total" to represent non-linear functions. $\endgroup$ – Dennis Soemers Aug 19 '18 at 19:45
4
$\begingroup$

Nonlinear relations between input and output can be achieved by using a nonlinear activation function on the value of each neuron, before it's passed on to the neurons in the next layer.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.