3
$\begingroup$

What is the simplest classification problem which cannot be solved by a perceptron (that is a single-layered feed-forward neural network, with no hidden layers and step activation function), but it can be solved by the same network if the activation function is swapped out to a differentiable activation function (e.g. sigmoid, tanh)?

In the first case, the training would be done with the perceptron training rule, in the second case with the delta rule.

Note that regression problems cannot be solved by perceptrons, so I'm interested in classification only.

$\endgroup$
1
  • 1
    $\begingroup$ Anything with a non-step output, perhaps? I thought that a perceptron with a different activation function is still a perceptron. $\endgroup$
    – user253751
    Mar 13 '20 at 10:42
1
$\begingroup$

Anything that is not linearly separable cant be solved perceptrons, unless you use feature maps on data to map them to a higher dimension in which it is linearly separable.

As a simple, concrete example, perceptron cant learn the XOR function.

This page might help you further.

$\endgroup$
2
  • 1
    $\begingroup$ Someone had already provided an answer to this question in the past, but it was deleted by the author after I commented saying that they were not addressing this part "but it can be solved by the same network if the activation function is swapped out to a differentiable activation function (e.g. sigmoid, tanh)?". Maybe you should address this part too. $\endgroup$
    – nbro
    Mar 22 '20 at 20:33
  • $\begingroup$ I am not saying changing the activation function changes or not the set of problems that the perceptron can solve. I am just asking you to explain why. $\endgroup$
    – nbro
    Mar 22 '20 at 21:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.