7
$\begingroup$

As I know, a single layer neural network can only do linear operations, but multilayered ones can.

Also, I recently learned that finite matrices/tensors, which are used in many neural networks, can only represent linear operations.

However, multi-layered neural networks can represent non-linear (even much more complex than being just a nonlinear) operations.

What makes it happen? The activation layer?

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ The very quick answer is; yes, nonlinear activation functions in between linear transformations (matrix multiplications) allow for the "total" to represent non-linear functions. $\endgroup$
    – Dennis Soemers ♦
    Commented Aug 19, 2018 at 19:45

1 Answer 1

Reset to default
5
$\begingroup$

Nonlinear relations between input and output can be achieved by using a nonlinear activation function on the value of each neuron, before it's passed on to the neurons in the next layer.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .