7
There does not appear to be a historical consensus on this.
The Wikipedia page on the Perceptrons book (which does not come down on either side) gives an argument that the ability of MLPs to compute any Boolean function was widely known at the time (at the very least to McCulloch and Pitts).
However, this page gives an account by someone present at the MIT ...
4
It can be done.
The activation function of a neuron does not have to be monotonic. The activation that Rahul suggested can be implemented via a continuously differentiable function, for example $ f(s) = exp(-k(1-s)^2) $ which has a nice derivative $f'(s) = 2k~(1-s)f(s)$. Here, $s=w_0~x_0+w_1~x_1$. Therefore, standard gradient-based learning algorithms are ...
2
The main problems are that your activation function is not monotonic (as pointed out by csrev), and that it is not continuously differentiable. These make it very difficult / impossible to use standard gradient-based learning algorithms.
So yes, there may exist a good solution of weight values, but it is very difficult to find or approximate those weight ...
2
Indeed I think the problem is with the way you've defined the activation function. By selecting it arbitrarily, you could solve many specific problems. In practice, activation functions used are monotonic. It keeps the error function convex at a per-layer level. In theory though I'm not sure exactly what Rosenblatt has claimed so it might be worth calling ...
2
The section of the book Perceptrons: An Introduction to Computational Geometry (expanded edition, third printing, 1988) that shows the limitations of the perceptron should be 11.8 The Nonseparable Case (p. 181), where the authors write
There are many reasons for studying the operation of the perceptron learning program when there is no $\mathbf{A}^*$ with ...
2
The initialization of the weights has a big impact on the results. I'm not sure specifically for the XOR gate, but the error can have a local minimum that the network can get "stuck" in during training. Using stochastic gradient descent can help give some randomness that gets the error out of these pits. Also, for the sigmoid function, weights ...
2
Before proving that XOR cannot be linearly separable, we first need to prove a lemma:
Lemma 1
Lemma: If 3 points are collinear and the middle point has a different label than the other two, then these 3 points cannot be linearly separable.
Proof: Let us label the points as point $A$, $B$, and $C$. $A$ and $C$ have the same label, and $B$ has a different ...
1
Anything that is not linearly separable cant be solved perceptrons, unless you use feature maps on data to map them to a higher dimension in which it is linearly separable.
As a simple, concrete example, perceptron cant learn the XOR function.
This page might help you further.
1
2 perceptrons without bias (+1 in the output layer, to get the result as 1 number).
1
I'd bet, you're doing something wrong, though I can't tell what it is. Try to change the learning rate dynamically, try to train in varying order, ....
On the seconds thought, it looks like you're using the standard sigmoid function. Then you're doing it basically wrong. The input can only be exactly 1 if the input is infinite - or very big so that the ...
Only top voted, non community-wiki answers of a minimum length are eligible