Skip to main content
28 votes

Why is it believed that a single-layer perceptron can't solve XOR? Doesn't this example disprove that?

The perceptron has a step activation. This does not. But at a deeper level, what you are showing is that, if we define the neuron's activation function to be XOR (essentially what the mod 2 addition ...
chessprogrammer's user avatar
14 votes

Did Minsky and Papert know that multi-layer perceptrons could solve XOR?

Whether Minsky knew or not, it was definitely known to Rosenblatt, as he published those results in his really pioneering report - Principles of Neurodynamics: Perceptrons and the Theory of Brain ...
Cel's user avatar
  • 141
13 votes

Did Minsky and Papert know that multi-layer perceptrons could solve XOR?

There does not appear to be a historical consensus on this. The Wikipedia page on the Perceptrons book (which does not come down on either side) gives an argument that the ability of MLPs to compute ...
NietzscheanAI's user avatar
7 votes
Accepted

Why is the perceptron criterion function differentiable?

$\max(-y_i(w x_i), 0)$ is not partial derivable respect $w$ if $w x_i=0$. Loss functions are problematic when not derivable in some point, but even more when they are flat (constant) in some interval ...
pasaba por aqui's user avatar
7 votes

Did Minsky and Papert know that multi-layer perceptrons could solve XOR?

In section 13.2 Other Multilayer Machines (pp. 231-232) of the book Perceptrons: An Introduction to Computational Geometry (expanded edition, third printing, 1988) Minsky and Papert actually talk ...
nbro's user avatar
  • 40.9k
5 votes
Accepted

What is the significance of weights in a feedforward neural network?

You described a single-layer feedforward network. They can have multiple layers. The significance of the weights is that they make a linear transformation from the output of the previous layer and ...
Didami's user avatar
  • 391
4 votes
Accepted

Which Rosenblatt's paper describes Rosenblatt's perceptron training algorithm?

The paper (or report) that formally introduced the perceptron is The Perceptron — A Perceiving and Recognizing Automaton (1957) by Frank Rosenblatt. If you read the first page of this paper, you can ...
nbro's user avatar
  • 40.9k
4 votes
Accepted

Why can't the XOR linear inseparability problem be solved with one perceptron like this?

It can be done. The activation function of a neuron does not have to be monotonic. The activation that Rahul suggested can be implemented via a continuously differentiable function, for example $ f(s)...
Vladislav Gladkikh's user avatar
4 votes

Can a neuron have both a bias and a threshold?

I assume you're talking about a perceptron threshold function. One definition of it with an explicit threshold is $$f(\textbf{x})= \begin{cases} 1& \text{if } \textbf{w}\cdot\textbf{x} > t\\ 0&...
Philip Raeisghasem's user avatar
3 votes
Accepted

How do sigmoid functions make it so that the prediction $\hat{y}$ indicates the probability that the observed value, $y$, is $1$?

I am specifically asking about the probability that the value is 1 (that is, how sigmoid functions specifically check for this). They don't in general. In the quoted text, there is an explicit ...
Neil Slater's user avatar
  • 32.7k
3 votes

Is there a proof to explain why XOR cannot be linearly separable?

Here is a similar contradiction based answer using basic coordinate geometry. Is there a proof to explain why $XOR$ cannot be linearly separable? Let us suppose, if possible, that the $XOR$ function,...
Toshant Narula's user avatar
3 votes
Accepted

Is there a proof to explain why XOR cannot be linearly separable?

Before proving that XOR cannot be linearly separable, we first need to prove a lemma: Lemma 1 Lemma: If 3 points are collinear and the middle point has a different label than the other two, then ...
user3667125's user avatar
  • 1,600
3 votes

Why is the perceptron criterion function differentiable?

Since we're dealing with real-values variables, it is almost certainly the case that the argument of the function will not be $0$. If you care strongly about that point, you can just use sub-gradients ...
Robby Goetschalckx's user avatar
3 votes

How do two perceptrons produce different linear decision boundaries when learning?

I think your confusion comes from the fact that you are calling those two hidden nodes "perceptrons". You shouldn't call the hidden nodes in your network perceptrons. You should call them "nodes" or "...
nbro's user avatar
  • 40.9k
3 votes

Why can't the XOR linear inseparability problem be solved with one perceptron like this?

The main problems are that your activation function is not monotonic (as pointed out by csrev), and that it is not continuously differentiable. These make it very difficult / impossible to use ...
Dennis Soemers's user avatar
  • 10.4k
3 votes
Accepted

An explanation involving the sign activation, its affect on the loss function, and the perceptron and perceptron criterion: what is this saying?

sign is not continuous and not differentiable. Let's say it is defined as follows: $$ \text{sing}(a) = \begin{cases} +1 & \text{if $a>0$}\\ -1 &...
Aray Karjauv's user avatar
3 votes

What are (all) the differences between a neuron and a perceptron?

In addition to those mentioned differences, a perceptron can be thought of as a standalone model (which is trained with a specific algorithm, the perceptron algorithm), while the artificial neuron (...
nbro's user avatar
  • 40.9k
3 votes
Accepted

What's the difference between a "perceptron" and a GLM?

The perceptron uses the Heaviside step (or sign) function as the activation function (so you are not free to use any activation function), while a GLM is a generalization of linear regression, where ...
nbro's user avatar
  • 40.9k
3 votes
Accepted

Direct formula for calculating the optimum matrix which minimizes the perceptron error

The idea is correct, the last formula is wrong. In general $X$ will not be square, usually one has much more data than parameters. The data points will also be in general position, so that $X$ has ...
Lutz Lehmann's user avatar
2 votes

Perceptron learning algorithm: different accuracies for different training methods

In Brief: re-train your dataset. I believe where you get lower accuracy scores, your model has not converged to the final state. duplicate your dataset multiple times and create a bigger one, then ...
Alireza's user avatar
  • 405
2 votes

Why can't the XOR linear inseparability problem be solved with one perceptron like this?

Indeed I think the problem is with the way you've defined the activation function. By selecting it arbitrarily, you could solve many specific problems. In practice, activation functions used are ...
csrev's user avatar
  • 31
2 votes
Accepted

Why can't MLPs perform non-linear regression and classification?

In Single Perceptron / Multi-layer Perceptron(MLP), we only have linear separability because they are composed of input and output layers(some hidden layers in MLP) This is wrong. A multi-layer ...
nbro's user avatar
  • 40.9k
2 votes
Accepted

Why doesn't the set $\{ -2, +2 \}$ in $E(X) = (y − \text{sign}\{\overline{W} \cdot \overline{X} \}) \in \{ −2, +2 \}$ include $0$?

It is important to note that the exact statement is the eqation given below can never be 0 for misclassified points in $ S^+$ $$ E(X) = (y - \text{sign}\{\overline{W} \cdot \overline{X}\}) $$ And $S+$ ...
ssh's user avatar
  • 61
2 votes

Which part of "Perceptrons: An Introduction to Computational Geometry" tells that a perceptron cannot solve the XOR problem?

The section of the book Perceptrons: An Introduction to Computational Geometry (expanded edition, third printing, 1988) that shows the limitations of the perceptron should be 11.8 The Nonseparable ...
nbro's user avatar
  • 40.9k
2 votes
Accepted

Understanding the perceptron algorithm in the book "A Course in Machine Learning"

Y is the desired output of the perceptron (often referred to as target) , for the given set of input vectors. Rationale behind Y.a<=0 : Prerequisite knowledge : A=A-B : Moves vector A away from ...
Programmer's user avatar
2 votes
Accepted

What are the labels in figure1 in the Paper "The perceptron: A probabilistic model for information storage and organization in the brain"?

Circles: RETINA / $A_I$ (POJECTION AREA) / $A_{II}$ (ASSOCIATION AREA) Labels: (LOCALISED CONNECTIONS) / (RANDOM CONNECTIONS) / (RANDOM CONNECTIONS) again / RESPONSES
Oliver Mason's user avatar
  • 5,397
2 votes

Direct formula for calculating the optimum matrix which minimizes the perceptron error

As you understand, $E$ is the definition of loss function. This function defines square of the difference between weights applied to $X_i$, namely output of the perception, and $Y_i$ the desired ...
OmG's user avatar
  • 1,826
1 vote

Where does the so-called 'loss' / 'loss function' fit into the idea of a perceptron / artificial neuron (as presented in the figure)?

Assume we have a binary classification problem, which we want to solve with a simple single-layer perceptron. For a 2d space, a perceptron will have 2 inputs $x_1$ and $x_2$, and a bias denoted $x_0$ ...
Aray Karjauv's user avatar
1 vote

Where does the so-called 'loss' / 'loss function' fit into the idea of a perceptron / artificial neuron (as presented in the figure)?

The loss function is simply a way to measure how wrong a neural network is, it doesn't affect the output of the neuron. Say we have a neural network with 3 output neurons that attempts to classify ...
Unnamed's user avatar
  • 111
1 vote
Accepted

Where does the so-called 'loss' / 'loss function' fit into the idea of a perceptron / artificial neuron (as presented in the figure)?

Loss function is a function used to measure the loss. It is not used in any component of a neuron. It is used in updating the weights of the neuron i.e., in order to train the neuron. The contribution ...
hanugm's user avatar
  • 3,890

Only top scored, non community-wiki answers of a minimum length are eligible