2
$\begingroup$

A single neuron with 2 weights and identity activation can learn addition/subtraction as the 2 weights will converge to 1 and 1 (addition), or 1 and -1 (subtraction).

However, for multiplication and division, it's not that easy. Can a single neuron learn multiplication or division? If not, how many layers of DNN can learn these?

$\endgroup$

1 Answer 1

3
$\begingroup$

In reallity any continous function on a compact can be approximated by a neural network having one hidden layer with a finite number of neurones (This is the Universal Approximation Theorem). Thus you only need one hidden layer to approximate the multiplication on a compact, note that you need to apply a non linear activation on the hidden layer to do this.

$\endgroup$
2
  • $\begingroup$ A Neural network cannot approximate an unbounded function. $\endgroup$
    – user9947
    Jan 3, 2020 at 19:09
  • $\begingroup$ All functions are bounded on a compact $\endgroup$
    – hola
    Jan 3, 2020 at 19:44

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .