Questions tagged [perceptron]
For questions about the perceptron learning algorithm in Machine Learning.
48
questions
0
votes
1answer
25 views
How to show $\rho > 0$ when $\rho$ be minimum attainable from $y_n(W^{*T}X_n)$, where $W^*$ the vector that separates the data?
In the book Learning from Data written (by Abu Mostafa), we have the following exercise:
Let $\rho$ be minimum attainable from $y_n(W^{*T}X_n)$ where $W^*$ is the vector that separates the data. Show ...
3
votes
1answer
56 views
What's the difference between a "perceptron" and a GLM?
In a comment to this question user nbro comments:
As a side note, "perceptrons" and "neural networks" may not be the same thing. People usually use the term perceptron to refer to ...
0
votes
0answers
19 views
What's the effect of increasing hidden nodes?
Topic Demarcation
I find many topics on "how to choose the number of hidden nodes". I'm not interested in the answer to that question.
What I learned
I learned, that if you increase the ...
1
vote
0answers
24 views
Is the main difference between the logistic regression and the perceptron the activation function they use?
I went through a Stats StackExchange's post about the difference between logistic regression and perceptron, which is too long to get the key point.
I'd like to consider the question in terms of the ...
1
vote
1answer
55 views
What are (all) the differences between a neuron and a perceptron?
I know two differences between a neuron and a perceptron
Neuron employs non-linear activation function and perceptron employs only a threshold activation function.
Output of a neuron is not ...
1
vote
1answer
27 views
Could someone help tell what the labels are pointed out by red rectangles?
The following figure comes from the paper The perceptron: A probabilistic model for information storage and organization in the brain
I can tell the labels pointed out by blue rectangles are: "...
0
votes
0answers
35 views
An explanation involving the sign activation, its affect on the loss function, and the perceptron and perceptron criterion: what is this saying? (#2)
I recently asked a very similar question here, but the answer only seems to address the first part of the quote, rather than the second part that contains the perceptron criterion example. Therefore, ...
1
vote
1answer
21 views
What are the labels in figure1 in the Paper "The perceptron: A probabilistic model for information storage and organization in the brain"?
This figure
comes from The perceptron: A probabilistic model for information storage and organization in the brain
I guess the first circle (neuron) labels RETINA, the second labels perceptron area, ...
1
vote
0answers
28 views
Is the formula $\frac {1}{s}\sum _{j=1}^{s}|d_{j}-y_{j}(t)|$ the correct form of 0-1 loss function, in the context of Perceptron?
Per page 7 of this MIT lecture notes, the original single-layer Perceptron uses 0-1 loss function.
Wikipedia uses
$${\displaystyle {\frac {1}{s}}\sum _{j=1}^{s}|d_{j}-y_{j}(t)|} \tag{1}$$
to denote ...
0
votes
1answer
55 views
An explanation involving the sign activation, its affect on the loss function, and the perceptron and perceptron criterion: what is this saying?
I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. Chapter 1.2.1.3 Choice of Activation and Loss Functions says the following:
The classical activation ...
2
votes
1answer
41 views
Is $(y_i - \hat y_i)x_i$, part of the formula for updating weights for perceptron, the gradient of some kind of loss function?
A post gives a formula for perceptron to update weights
I understand almost all the parts of it, except for the part $(y_i - \hat y_i)x_i$ where does it come from? Is it the gradient of some kind of ...
1
vote
1answer
51 views
Flowchart for a simplified perceptron leaning algorithm [critique request]
I made a flowchart for a simplified perceptron leaning algorithm.
Here is the process of the leaning algorithm.
Step_1: Initialize the weights first.
Step_2: Get a training example randomly and make ...
0
votes
0answers
24 views
Incorporating regularization for the kernel perceptron
To my understanding, the following is how the kernel perceptron works.
Kernel perceptron algorithm
The parameters to be calculated are $\alpha = \begin{pmatrix} \alpha_1 &\ldots &\...
2
votes
0answers
44 views
Backpropagation not working as expected
I'm new to neural networks and I try to make a model that is guessing if a point is below or above relative to a function output. The idea is inspired from this video https://youtu.be/DGxIcDjPzac .
...
3
votes
3answers
239 views
Where does the so-called 'loss' / 'loss function' fit into the idea of a perceptron / artificial neuron (as presented in the figure)?
I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. Chapter 1.2.1.3 Choice of Activation and Loss Functions presents the following figure:
$\overline{X}$ is ...
2
votes
1answer
62 views
How do sigmoid functions make it so that the prediction $\hat{y}$ indicates the probability that the observed value, $y$, is $1$?
I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. Chapter 1.2.1.3 Choice of Activation and Loss Functions says the following:
The choice of activation ...
1
vote
0answers
27 views
Is it possible to train a perceptron to tell if a picture is a dog or cat?
I know perceptron is a linear classifier that tells linearly separable binary class data, such as iris setosa vs. iris versicolor via their sepal's length and width.
I'd just like to know if I have 2 ...
1
vote
1answer
46 views
In practice, are perceptrons typically implemented as objects?
I'm fairly new to ANNs. I know the general structure, the math, and the algorithms behind them. I figured the logical next step on my journey to fully understanding them would to be implement one ...
0
votes
0answers
43 views
What are the math theorems regarding the Multilayer Perceptron?
I've come across a theorem "Convergence theorem
Simple Perceptron" for the first time, here-> https://zaguan.unizar.es/record/69205/files/TAZ-TFG-2018-148.pdf, page 27, (is in Spanish)
...
5
votes
1answer
144 views
Why did the developement of neural networks stop between 50s and 80s?
In a video lecture on the development of neural networks and the history of deep learning (you can start from minute 13), the lecturer (Yann LeCunn) said that the development of neural networks ...
1
vote
0answers
32 views
What is the smallest upper bound for a number of functions in a range that are computable by a perceptron?
I'm reading this book chapter, and I'm looking at the questions on the last page. Can someone explain question 2 on the last page to me, or show me an example of a solution so I can understand it?
The ...
0
votes
0answers
19 views
What are the general inequalities needed for the logic gate perceptrons?
I'm trying to understand how the logic gates (e.g. AND, OR, NOT, NAND) can be built into single-layer perceptrons.
I understand specific examples of weights and thresholds for the gates, but I'm stuck ...
1
vote
1answer
101 views
What is the equation to update the weights in the perceptron algorithm?
I'm trying to understand the solution to question 4 of this midterm paper.
The question and solution is as follows:
I thought that the process for updating weights was:
...
2
votes
2answers
902 views
Is there a proof to explain why XOR cannot be linearly separable?
Can someone explain to me with a proof or example why you can't linearly separate XOR (and therefore need a neural network, the context I'm looking at it in)?
I understand why it's not linearly ...
5
votes
1answer
307 views
How should we interpret this figure that relates the perceptron criterion and the hinge loss?
I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. Chapter 1.2.1.2 Relationship with Support Vector Machines says the following:
The perceptron criterion is ...
2
votes
1answer
94 views
Why doesn't the set $\{ -2, +2 \}$ in $E(X) = (y − \text{sign}\{\overline{W} \cdot \overline{X} \}) \in \{ −2, +2 \}$ include $0$?
I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. Chapter 1.2.1.2 Relationship with Support Vector Machines says the following:
The perceptron criterion is ...
8
votes
2answers
221 views
Why is the perceptron criterion function differentiable?
I'm reading chapter one of the book called Neural Networks and Deep Learning from Aggarwal.
In section 1.2.1.1 of the book, I'm learning about the perceptron. One thing that book says is, if we use ...
0
votes
1answer
632 views
Why can't MLPs perform non-linear regression and classification?
In this page it's told:
In Single Perceptron / Multi-layer Perceptron(MLP), we only have linear separability because they are composed of input and output layers(some hidden layers in MLP)
What ...
1
vote
0answers
27 views
How can I adapt this script (to test the robustness of a perceptron) to test the robustness of a multi-layer perceptron?
The following script is from Trappenberg's Fundamentals of Computational Neuroscience and is used to test a perceptron's robustness against noise.
However, how would one alter it to test the output ...
3
votes
1answer
320 views
What is the simplest classification problem which cannot be solved by a perceptron?
What is the simplest classification problem which cannot be solved by a perceptron (that is a single-layered feed-forward neural network, with no hidden layers and step activation function), but it ...
2
votes
1answer
126 views
Is there a mathematical theory behind why MLP can classify handwritten digits?
I'm trying to really understand how multi-layer perceptrons work. I want to prove mathematically that MLP's can classify handwritten digits. The only thing I really have is that each perceptron can ...
3
votes
1answer
87 views
Which part of "Perceptrons: An Introduction to Computational Geometry" tells that a perceptron cannot solve the XOR problem?
In the book "Perceptrons: An Introduction to Computational Geometry" by Minsky and Papert (1969), which part of this book tells that a single-layer perceptron could not solve the XOR problem?
I have ...
3
votes
1answer
58 views
How do I determine the most appropriate classifier for a certain problem?
Consider a Bayesian classifier used in spam e-mail filtering. It converts an e-mail to a vector, most of the time using the bag-of-words method. Although it learns first before getting employed, it ...
3
votes
1answer
226 views
What are the reasons a perceptron is not able to learn?
I'm just starting to learn about neural networking and I decided to study a simple 3-input perceptron to get started with. I am also only using binary inputs to gain a full understanding of how the ...
2
votes
1answer
56 views
Can a neuron have both a bias and a threshold?
I have not seen a neuron that uses both a bias and a threshold. Why is this?
2
votes
1answer
234 views
How do two perceptrons produce different linear decision boundaries when learning?
I've learned that you can use two perceptrons to ultimately create a classifier for non-linearly separable data. I'm trying to understand how / if these two perceptrons converge to two different ...
1
vote
1answer
225 views
A neural network for digits recognition doesn't work (MNIST, Numpy) [closed]
I'm a beginner in machine learning and I was trying to make a test neural network for digits recognition from scratch using Numpy. I used MNIST dataset for training and testing. Input layer have 28*28 ...
3
votes
0answers
33 views
Batch PTA stopping condition
I am reviewing my Neural Network lectures and I have a doubt: My book's (Haykin) batch PTA describes a cost function which is defined over the set of the misclassified inputs.
I have always been ...
2
votes
0answers
162 views
If we use a perceptron with a non-monotonic activation function, can it solve the XOR problem?
I found several papers about how to build a perceptron able to solve the XOR problem. The papers describe a solution where the heaviside step function is replaced by a non-monotonic activation ...
5
votes
5answers
675 views
Why can't the XOR linear inseparability problem be solved with one perceptron like this?
Consider a perceptron where $w_0=1$ and $w_1=1$:
Now, suppose that we use the following activation function
\begin{align}
f(x)=
\begin{cases}
1, \text{ if }x =1\\
0, \text{ otherwise}
\end{cases}
\...
6
votes
1answer
430 views
Which Rosenblatt's paper describes Rosenblatt's perceptron training algorithm?
I struggle to find Rosenblatt's perceptron training algorithm in any of his publications from 1957 - 1961, namely:
Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms
The ...
2
votes
1answer
200 views
Is the bias supposed to be updated in the perceptron learning algorithm?
I am using the following perceptron formula $\text{step}\left(\sum(w_ix_i)-\theta \right)$.
Is $\theta$ supposed to be updated in a perceptron, like the weights $w_i$? If so, what is the formula for ...
1
vote
1answer
228 views
Minimum number of perceptrons for an n-bit truth table?
Suppose I have a Boolean function that maps N bits to one bit. If I understand correctly, this function will have $2^{2^N}$ possible configurations of its truth table.
What is the minimum number of ...
0
votes
2answers
282 views
Creating a working perceptron
I'm trying to learn more about AI by trying to program a neural network. First I'm trying to understand writing my own perceptron but I'm struggling to get a basic perceptron working correctly.
I've ...
3
votes
2answers
2k views
Perceptron learning algorithm: different accuracies for different training methods
So, my question is a bit theoretical. I have been trying to implement a perceptron based classifier with outputs 1 and 0 depending on the category. I have used 2 methods: The ...
9
votes
2answers
3k views
What are the main differences between a perceptron and a naive Bayes classifier?
What are the main differences between a perceptron and a naive Bayes classifier?
3
votes
1answer
279 views
Understanding the perceptron algorithm in the book "A Course in Machine Learning"
The following text is from Hal Daumé III's "A Course in Machine Learning" online text book (Page-41).
I understand that $D$ is the size of the input vector $x$.
What is $y$? Why is it introduced in ...
8
votes
2answers
2k views
Did Minsky and Papert know that multi-layer perceptrons could solve XOR?
In their famous book entitled Perceptrons: An Introduction to Computational Geometry, Minsky and Papert show that a perceptron can't solve the XOR problem. This contributed to the first AI winter, ...
