As I know, a single layer neural network can only do linear operations, but multilayered ones can.

Also, I recently learned that finite matrices/tensors, which are used in many neural networks, can only represent linear operations.

However, multi-layered neural networks can represent non-linear (even much more complex than being just a nonlinear) operations.

What makes it happen? The activation layer?

  • 2
    $\begingroup$ The very quick answer is; yes, nonlinear activation functions in between linear transformations (matrix multiplications) allow for the "total" to represent non-linear functions. $\endgroup$
    – Dennis Soemers
    Aug 19, 2018 at 19:45

1 Answer 1


Nonlinear relations between input and output can be achieved by using a nonlinear activation function on the value of each neuron, before it's passed on to the neurons in the next layer.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .