12
votes
Did Minsky and Papert know that multi-layer perceptrons could solve XOR?
There does not appear to be a historical consensus on this.
The Wikipedia page on the Perceptrons book (which does not come down on either side) gives an argument that the ability of MLPs to compute ...
- 7,226
10
votes
Did Minsky and Papert know that multi-layer perceptrons could solve XOR?
Whether Minsky knew or not, it was definitely known to Rosenblatt, as he published those results in his really pioneering report - Principles of Neurodynamics: Perceptrons and the Theory of Brain ...
- 101
7
votes
Accepted
Why is the perceptron criterion function differentiable?
$\max(-y_i(w x_i), 0)$ is not partial derivable respect $w$ if $w x_i=0$.
Loss functions are problematic when not derivable in some point, but even more when they are flat (constant) in some interval ...
- 1,283
7
votes
Did Minsky and Papert know that multi-layer perceptrons could solve XOR?
In section 13.2 Other Multilayer Machines (pp. 231-232) of the book Perceptrons: An Introduction to Computational Geometry (expanded edition, third printing, 1988) Minsky and Papert actually talk ...
- 39.5k
5
votes
Accepted
What is the significance of weights in a feedforward neural network?
You described a single-layer feedforward network. They can have multiple layers. The significance of the weights is that they make a linear transformation from the output of the previous layer and ...
- 381
4
votes
Accepted
Which Rosenblatt's paper describes Rosenblatt's perceptron training algorithm?
The paper (or report) that formally introduced the perceptron is The Perceptron — A Perceiving and Recognizing Automaton (1957) by Frank Rosenblatt. If you read the first page of this paper, you can ...
- 39.5k
4
votes
Accepted
Why can't the XOR linear inseparability problem be solved with one perceptron like this?
It can be done.
The activation function of a neuron does not have to be monotonic. The activation that Rahul suggested can be implemented via a continuously differentiable function, for example $ f(s)...
4
votes
Can a neuron have both a bias and a threshold?
I assume you're talking about a perceptron threshold function. One definition of it with an explicit threshold is
$$f(\textbf{x})=
\begin{cases}
1& \text{if } \textbf{w}\cdot\textbf{x} > t\\
0&...
- 2,028
3
votes
Accepted
How do sigmoid functions make it so that the prediction $\hat{y}$ indicates the probability that the observed value, $y$, is $1$?
I am specifically asking about the probability that the value is 1 (that is, how sigmoid functions specifically check for this).
They don't in general. In the quoted text, there is an explicit ...
- 30.2k
3
votes
Is there a proof to explain why XOR cannot be linearly separable?
Here is a similar contradiction based answer using basic coordinate geometry.
Is there a proof to explain why $XOR$ cannot be linearly separable?
Let us suppose, if possible, that the $XOR$ function,...
3
votes
Accepted
Is there a proof to explain why XOR cannot be linearly separable?
Before proving that XOR cannot be linearly separable, we first need to prove a lemma:
Lemma 1
Lemma: If 3 points are collinear and the middle point has a different label than the other two, then ...
- 1,510
3
votes
Why is the perceptron criterion function differentiable?
Since we're dealing with real-values variables, it is almost certainly the case that the argument of the function will not be $0$.
If you care strongly about that point, you can just use sub-gradients ...
3
votes
How do two perceptrons produce different linear decision boundaries when learning?
I think your confusion comes from the fact that you are calling those two hidden nodes "perceptrons". You shouldn't call the hidden nodes in your network perceptrons. You should call them "nodes" or "...
- 39.5k
3
votes
Why can't the XOR linear inseparability problem be solved with one perceptron like this?
The main problems are that your activation function is not monotonic (as pointed out by csrev), and that it is not continuously differentiable. These make it very difficult / impossible to use ...
- 10.1k
3
votes
Accepted
An explanation involving the sign activation, its affect on the loss function, and the perceptron and perceptron criterion: what is this saying?
sign is not continuous and not differentiable. Let's say it is defined as follows:
$$
\text{sing}(a) = \begin{cases}
+1 & \text{if $a>0$}\\
-1 &...
- 927
3
votes
What are (all) the differences between a neuron and a perceptron?
In addition to those mentioned differences, a perceptron can be thought of as a standalone model (which is trained with a specific algorithm, the perceptron algorithm), while the artificial neuron (...
- 39.5k
3
votes
Accepted
What's the difference between a "perceptron" and a GLM?
The perceptron uses the Heaviside step (or sign) function as the activation function (so you are not free to use any activation function), while a GLM is a generalization of linear regression, where ...
- 39.5k
2
votes
Perceptron learning algorithm: different accuracies for different training methods
In Brief:
re-train your dataset. I believe where you get lower accuracy scores, your model has not converged to the final state. duplicate your dataset multiple times and create a bigger one, then ...
- 405
2
votes
Why can't the XOR linear inseparability problem be solved with one perceptron like this?
Indeed I think the problem is with the way you've defined the activation function. By selecting it arbitrarily, you could solve many specific problems. In practice, activation functions used are ...
- 31
2
votes
Accepted
Why can't MLPs perform non-linear regression and classification?
In Single Perceptron / Multi-layer Perceptron(MLP), we only have linear separability because they are composed of input and output layers(some hidden layers in MLP)
This is wrong.
A multi-layer ...
- 39.5k
2
votes
Accepted
Why doesn't the set $\{ -2, +2 \}$ in $E(X) = (y − \text{sign}\{\overline{W} \cdot \overline{X} \}) \in \{ −2, +2 \}$ include $0$?
It is important to note that the exact statement is the eqation given below can never be 0 for misclassified points in $ S^+$
$$ E(X) = (y - \text{sign}\{\overline{W} \cdot \overline{X}\}) $$
And $S+$ ...
- 61
2
votes
Which part of "Perceptrons: An Introduction to Computational Geometry" tells that a perceptron cannot solve the XOR problem?
The section of the book Perceptrons: An Introduction to Computational Geometry (expanded edition, third printing, 1988) that shows the limitations of the perceptron should be 11.8 The Nonseparable ...
- 39.5k
2
votes
Accepted
Understanding the perceptron algorithm in the book "A Course in Machine Learning"
Y is the desired output of the perceptron (often referred to as target) , for the given set of input vectors.
Rationale behind Y.a<=0 :
Prerequisite knowledge :
A=A-B : Moves vector A away from ...
- 164
2
votes
Accepted
What are the labels in figure1 in the Paper "The perceptron: A probabilistic model for information storage and organization in the brain"?
Circles: RETINA / $A_I$ (POJECTION AREA) / $A_{II}$ (ASSOCIATION AREA)
Labels: (LOCALISED CONNECTIONS) / (RANDOM CONNECTIONS) / (RANDOM CONNECTIONS) again / RESPONSES
- 5,352
1
vote
Where does the so-called 'loss' / 'loss function' fit into the idea of a perceptron / artificial neuron (as presented in the figure)?
Assume we have a binary classification problem, which we want to solve with a simple single-layer perceptron. For a 2d space, a perceptron will have 2 inputs $x_1$ and $x_2$, and a bias denoted $x_0$ ...
- 927
1
vote
Where does the so-called 'loss' / 'loss function' fit into the idea of a perceptron / artificial neuron (as presented in the figure)?
The loss function is simply a way to measure how wrong a neural network is, it doesn't affect the output of the neuron.
Say we have a neural network with 3 output neurons that attempts to classify ...
- 11
1
vote
Accepted
Where does the so-called 'loss' / 'loss function' fit into the idea of a perceptron / artificial neuron (as presented in the figure)?
Loss function is a function used to measure the loss. It is not used in any component of a neuron. It is used in updating the weights of the neuron i.e., in order to train the neuron.
The contribution ...
- 3,612
1
vote
Accepted
In practice, are perceptrons typically implemented as objects?
Unless one performs an exhaustive search, it's difficult to answer your question.
However, in the widely used libraries, such as TensorFlow, PyTorch and sklearn, most abstractions (like neural ...
- 39.5k
1
vote
Accepted
Why did the developement of neural networks stop between 50s and 80s?
I will first address your main question "Why did the development of neural networks stop between 50s and 80s?"
In 40-50s there was a lot of progress (McCulloch and Pitts); the perceptron was ...
- 264
1
vote
Accepted
What is the equation to update the weights in the perceptron algorithm?
I will tell you my knowledge, correct me if I am wrong.
Perceptron Learning Algorithm (PLA) is a simple method to solve the binary classification problem.
Define a function:
$$
f_w(x) = w^Tx + b
$$
...
- 902
Only top scored, non community-wiki answers of a minimum length are eligible