3
$\begingroup$

What is the simplest classification problem which cannot be solved by a perceptron (that is a single-layered feed-forward neural network, with no hidden layers and step activation function), but it can be solved by the same network if the activation function is swapped out to a differentiable activation function (e.g. sigmoid, tanh)?

In the first case, the training would be done with the perceptron training rule, in the second case with the delta rule.

Note that regression problems cannot be solved by perceptrons, so I'm interested in classification only.

$\endgroup$
1
  • 1
    $\begingroup$ Anything with a non-step output, perhaps? I thought that a perceptron with a different activation function is still a perceptron. $\endgroup$
    – user253751
    Mar 13, 2020 at 10:42

1 Answer 1

1
$\begingroup$

Anything that is not linearly separable cant be solved perceptrons, unless you use feature maps on data to map them to a higher dimension in which it is linearly separable.

As a simple, concrete example, perceptron cant learn the XOR function.

This page might help you further.

$\endgroup$
2
  • 1
    $\begingroup$ Someone had already provided an answer to this question in the past, but it was deleted by the author after I commented saying that they were not addressing this part "but it can be solved by the same network if the activation function is swapped out to a differentiable activation function (e.g. sigmoid, tanh)?". Maybe you should address this part too. $\endgroup$
    – nbro
    Mar 22, 2020 at 20:33
  • $\begingroup$ I am not saying changing the activation function changes or not the set of problems that the perceptron can solve. I am just asking you to explain why. $\endgroup$
    – nbro
    Mar 22, 2020 at 21:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .