2
$\begingroup$

A single neuron with 2 weights and identity activation can learn addition/subtraction as the 2 weights will converge to 1 and 1 (addition), or 1 and -1 (subtraction).

However, for multiplication and division, it's not that easy. Can a single neuron learn multiplication or division? If not, how many layers of DNN can learn these?

$\endgroup$
3
$\begingroup$

In reallity any continous function on a compact can be approximated by a neural network having one hidden layer with a finite number of neurones (This is the Universal Approximation Theorem). Thus you only need one hidden layer to approximate the multiplication on a compact, note that you need to apply a non linear activation on the hidden layer to do this.

| improve this answer | |
$\endgroup$
  • $\begingroup$ A Neural network cannot approximate an unbounded function. $\endgroup$ – DuttaA Jan 3 at 19:09
  • $\begingroup$ All functions are bounded on a compact $\endgroup$ – hola Jan 3 at 19:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.