Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [perceptron]

For questions about the perceptron learning algorithm in Machine Learning.

Filter by
Sorted by
Tagged with
0 votes
0 answers
16 views

Trouble understanding why adaline works

Recently came across adaline (an improvement on perceptron) but I am having trouble understanding why adaline works. Lets take an example of 2D binary classification task. Assume line 'l' is a linear ...
Anjusha C's user avatar
1 vote
2 answers
93 views

Direct formula for calculating the optimum matrix which minimizes the perceptron error

Suppose we have a perceptron without bias and $f(x) = x$ as activation function and matrices $X,Y,W$ that input training data are columns of matrix $X$, $Y$ is targets matrix (columns are ordered with ...
hasanghaforian's user avatar
3 votes
2 answers
5k views

Why is it believed that a single-layer perceptron can't solve XOR? Doesn't this example disprove that?

...
Jaroslav Tavgen's user avatar
0 votes
1 answer
65 views

Is the Adaline just a Perceptron that uses Linear activation function and MSE cost function, and all the rest of steps are the same of Perceptron?

https://en.wikipedia.org/wiki/ADALINE https://pt.wikipedia.org/wiki/Perceptron I have a doubt about this: is the Adaline just a Perceptron that uses Linear activation function and MSE cost function, ...
will The J's user avatar
0 votes
0 answers
12 views

Implement perceptron given decision boundary function

Suppose we have a linear classifier for the classes ω1 and ω2 with characteristics vectors Xa=[a a]^T and Xb=[-a -a]^T correspondingly. Also suppose that the decision boundary that is defined by ...
TheExtraSpicyBeef's user avatar
1 vote
1 answer
209 views

Why does sklearn perceptron converge for linearly inseparable data points?

I learned that the perceptron algorithm only converges if the dataset is linearly separable. I am implementing this algorithm using scikit learn. The blue and orange points are from the training set, ...
jacquesadit00's user avatar
0 votes
1 answer
94 views

How does a sigmoid neuron act like a perceptron in this scenario?

I have been reading Michael Nielsen’s book online on his website at http://neuralnetworksanddeeplearning.com/chap1.html. I am struggling to understand the second exercise: When c approaches infinity, ...
QuantNoob's user avatar
1 vote
0 answers
81 views

What is the most accurate way of building a Perceptron using only NumPy?

For context, I am trying to write a bunch of neural network programs using no other packages besides NumPy for educational purposes. I am trying to make them as simple as possible, i.e. removing the ...
user avatar
3 votes
2 answers
257 views

Is there any variant of perceptron convergence algorithm that ensures uniqueness?

The perceptron convergence algorithm given below ensures the convergence of weights of the perceptron provided enough data points and iterations. Although it ensures convergence by finally getting a ...
hanugm's user avatar
  • 3,890
2 votes
0 answers
36 views

Was the original perceptron machine really just comparing images to the "average image for each class"?

I've been looking into the history of artificial neural networks, and only recently learned that the original Mark 1 Perceptron was only a single layer network. It would iteratively modify the ...
Seán Healy's user avatar
1 vote
1 answer
326 views

Why are today's neural networks not modeled with probability theory?

In the paper The Perceptron: A probabilistic model for information storage and organization in the brain, Rosenblatt used the probability theory to model his perceptron. My professor told me that ...
Collo's user avatar
  • 11
2 votes
0 answers
556 views

Did the unsolved XOR problem in "Perceptrons: An Introduction to Computational Geometry" 1969 book really cause the winter of the AI in 1974?

Winter of AI definition: periods of reduced funding and interest in artificial intelligence research, due to unmet expectations after a period of hype. There have been at least two major AI winters ...
rubengavidia0x's user avatar
0 votes
0 answers
45 views

Why isn't my perceptron having the expected costs?

I want to implement a single perceptron for linear regression using the following formulas: The input data for the first case is one column (x(392, 1); y(392, 1)) ...
Rim Sleimi's user avatar
0 votes
1 answer
56 views

How to show $\rho > 0$ when $\rho$ be minimum attainable from $y_n(W^{*T}X_n)$, where $W^*$ the vector that separates the data?

In the book Learning from Data written (by Abu Mostafa), we have the following exercise: Let $\rho$ be minimum attainable from $y_n(W^{*T}X_n)$ where $W^*$ is the vector that separates the data. Show ...
John Rawls's user avatar
5 votes
1 answer
367 views

What's the difference between a "perceptron" and a GLM?

In a comment to this question user nbro comments: As a side note, "perceptrons" and "neural networks" may not be the same thing. People usually use the term perceptron to refer to ...
R.M.'s user avatar
  • 308
1 vote
0 answers
76 views

Is the main difference between the logistic regression and the perceptron the activation function they use?

I went through a Stats StackExchange's post about the difference between logistic regression and perceptron, which is too long to get the key point. I'd like to consider the question in terms of the ...
JJJohn's user avatar
  • 221
2 votes
1 answer
3k views

What are (all) the differences between a neuron and a perceptron?

I know two differences between a neuron and a perceptron Neuron employs non-linear activation function and perceptron employs only a threshold activation function. The output of a neuron is not ...
hanugm's user avatar
  • 3,890
1 vote
1 answer
33 views

Could someone help tell what the labels are pointed out by red rectangles?

The following figure comes from the paper The perceptron: A probabilistic model for information storage and organization in the brain I can tell the labels pointed out by blue rectangles are: "...
JJJohn's user avatar
  • 221
1 vote
1 answer
64 views

What are the labels in figure1 in the Paper "The perceptron: A probabilistic model for information storage and organization in the brain"?

This figure comes from The perceptron: A probabilistic model for information storage and organization in the brain I guess the first circle (neuron) labels RETINA, the second labels perceptron area, ...
JJJohn's user avatar
  • 221
1 vote
0 answers
158 views

Is the formula $\frac {1}{s}\sum _{j=1}^{s}|d_{j}-y_{j}(t)|$ the correct form of 0-1 loss function, in the context of Perceptron?

Per page 7 of this MIT lecture notes, the original single-layer Perceptron uses 0-1 loss function. Wikipedia uses $${\displaystyle {\frac {1}{s}}\sum _{j=1}^{s}|d_{j}-y_{j}(t)|} \tag{1}$$ to denote ...
JJJohn's user avatar
  • 221
1 vote
1 answer
679 views

An explanation involving the sign activation, its affect on the loss function, and the perceptron and perceptron criterion: what is this saying?

I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. Chapter 1.2.1.3 Choice of Activation and Loss Functions says the following: The classical activation ...
The Pointer's user avatar
3 votes
2 answers
575 views

Is $(y_i - \hat y_i)x_i$, part of the formula for updating weights for perceptron, the gradient of some kind of loss function?

A post gives a formula for perceptron to update weights I understand almost all the parts of it, except for the part $(y_i - \hat y_i)x_i$ where does it come from? Is it the gradient of some kind of ...
JJJohn's user avatar
  • 221
2 votes
3 answers
467 views

Is my flowchart a good representation of the perceptron learning algorithm?

I made a flowchart for a simplified perceptron leaning algorithm. Here is the process of the learning algorithm. Initialize the weights first. Get a training example randomly and make a prediction. ...
JJJohn's user avatar
  • 221
1 vote
0 answers
111 views

Backpropagation not working as expected

I'm new to neural networks and I try to make a model that is guessing if a point is below or above relative to a function output. The idea is inspired from this video https://youtu.be/DGxIcDjPzac . ...
Valentin Stamate's user avatar
2 votes
3 answers
462 views

Where does the so-called 'loss' / 'loss function' fit into the idea of a perceptron / artificial neuron (as presented in the figure)?

I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. Chapter 1.2.1.3 Choice of Activation and Loss Functions presents the following figure: $\overline{X}$ is ...
The Pointer's user avatar
2 votes
1 answer
978 views

How do sigmoid functions make it so that the prediction $\hat{y}$ indicates the probability that the observed value, $y$, is $1$?

I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. Chapter 1.2.1.3 Choice of Activation and Loss Functions says the following: The choice of activation ...
The Pointer's user avatar
1 vote
0 answers
55 views

Is it possible to train a perceptron to tell if a picture is a dog or cat?

I know perceptron is a linear classifier that tells linearly separable binary class data, such as iris setosa vs. iris versicolor via their sepal's length and width. I'd just like to know if I have 2 ...
JJJohn's user avatar
  • 221
2 votes
1 answer
57 views

In practice, are perceptrons typically implemented as objects?

I'm fairly new to ANNs. I know the general structure, the math, and the algorithms behind them. I figured the logical next step on my journey to fully understanding them would to be implement one ...
lumpychum's user avatar
0 votes
0 answers
71 views

What are the math theorems regarding the Multilayer Perceptron?

I've come across a theorem "Convergence theorem Simple Perceptron" for the first time, here-> https://zaguan.unizar.es/record/69205/files/TAZ-TFG-2018-148.pdf, page 27, (is in Spanish) ...
Verónica Rmz.'s user avatar
5 votes
1 answer
367 views

Why did the developement of neural networks stop between 50s and 80s?

In a video lecture on the development of neural networks and the history of deep learning (you can start from minute 13), the lecturer (Yann LeCunn) said that the development of neural networks ...
Daviiid's user avatar
  • 575
1 vote
0 answers
54 views

What is the smallest upper bound for a number of functions in a range that are computable by a perceptron?

I'm reading this book chapter, and I'm looking at the questions on the last page. Can someone explain question 2 on the last page to me, or show me an example of a solution so I can understand it? The ...
Slowat_Kela's user avatar
0 votes
0 answers
147 views

What are the general inequalities needed for the logic gate perceptrons?

I'm trying to understand how the logic gates (e.g. AND, OR, NOT, NAND) can be built into single-layer perceptrons. I understand specific examples of weights and thresholds for the gates, but I'm stuck ...
Slowat_Kela's user avatar
1 vote
1 answer
3k views

What is the equation to update the weights in the perceptron algorithm?

I'm trying to understand the solution to question 4 of this midterm paper. The question and solution is as follows: I thought that the process for updating weights was: ...
Slowat_Kela's user avatar
6 votes
2 answers
10k views

Is there a proof to explain why XOR cannot be linearly separable?

Can someone explain to me with a proof or example why you can't linearly separate XOR (and therefore need a neural network, the context I'm looking at it in)? I understand why it's not linearly ...
Slowat_Kela's user avatar
7 votes
2 answers
895 views

How should we interpret this figure that relates the perceptron criterion and the hinge loss?

I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. Chapter 1.2.1.2 Relationship with Support Vector Machines says the following: The perceptron criterion is ...
The Pointer's user avatar
2 votes
1 answer
130 views

Why doesn't the set $\{ -2, +2 \}$ in $E(X) = (y − \text{sign}\{\overline{W} \cdot \overline{X} \}) \in \{ −2, +2 \}$ include $0$?

I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. Chapter 1.2.1.2 Relationship with Support Vector Machines says the following: The perceptron criterion is ...
The Pointer's user avatar
8 votes
2 answers
1k views

Why is the perceptron criterion function differentiable?

I'm reading chapter one of the book called Neural Networks and Deep Learning from Aggarwal. In section 1.2.1.1 of the book, I'm learning about the perceptron. One thing that book says is, if we use ...
Flávio Mendes's user avatar
1 vote
1 answer
3k views

Why can't MLPs perform non-linear regression and classification?

In this page it's told: In Single Perceptron / Multi-layer Perceptron(MLP), we only have linear separability because they are composed of input and output layers(some hidden layers in MLP) What ...
AleWolf's user avatar
  • 167
3 votes
1 answer
815 views

What is the simplest classification problem which cannot be solved by a perceptron?

What is the simplest classification problem which cannot be solved by a perceptron (that is a single-layered feed-forward neural network, with no hidden layers and step activation function), but it ...
yejaxey's user avatar
  • 31
2 votes
1 answer
147 views

Is there a mathematical theory behind why MLP can classify handwritten digits?

I'm trying to really understand how multi-layer perceptrons work. I want to prove mathematically that MLP's can classify handwritten digits. The only thing I really have is that each perceptron can ...
user8714896's user avatar
5 votes
1 answer
414 views

Which part of "Perceptrons: An Introduction to Computational Geometry" tells that a perceptron cannot solve the XOR problem?

In the book "Perceptrons: An Introduction to Computational Geometry" by Minsky and Papert (1969), which part of this book tells that a single-layer perceptron could not solve the XOR problem? I have ...
rimbaerl's user avatar
3 votes
1 answer
68 views

How do I determine the most appropriate classifier for a certain problem?

Consider a Bayesian classifier used in spam e-mail filtering. It converts an e-mail to a vector, most of the time using the bag-of-words method. Although it learns first before getting employed, it ...
user avatar
3 votes
1 answer
382 views

What are the reasons a perceptron is not able to learn?

I'm just starting to learn about neural networking and I decided to study a simple 3-input perceptron to get started with. I am also only using binary inputs to gain a full understanding of how the ...
James's user avatar
  • 131
1 vote
1 answer
131 views

Can a neuron have both a bias and a threshold?

I have not seen a neuron that uses both a bias and a threshold. Why is this?
hina munir's user avatar
3 votes
1 answer
775 views

How do two perceptrons produce different linear decision boundaries when learning?

I've learned that you can use two perceptrons to ultimately create a classifier for non-linearly separable data. I'm trying to understand how / if these two perceptrons converge to two different ...
ashar's user avatar
  • 39
0 votes
1 answer
310 views

A neural network for digits recognition doesn't work (MNIST, Numpy) [closed]

I'm a beginner in machine learning and I was trying to make a test neural network for digits recognition from scratch using Numpy. I used MNIST dataset for training and testing. Input layer have 28*28 ...
Андрій Немченко's user avatar
2 votes
0 answers
40 views

Batch PTA stopping condition

I am reviewing my Neural Network lectures and I have a doubt: My book's (Haykin) batch PTA describes a cost function which is defined over the set of the misclassified inputs. I have always been ...
Iacopo Olivo's user avatar
3 votes
0 answers
259 views

If we use a perceptron with a non-monotonic activation function, can it solve the XOR problem?

I found several papers about how to build a perceptron able to solve the XOR problem. The papers describe a solution where the heaviside step function is replaced by a non-monotonic activation ...
app_idea54's user avatar
6 votes
5 answers
1k views

Why can't the XOR linear inseparability problem be solved with one perceptron like this?

Consider a perceptron where $w_0=1$ and $w_1=1$: Now, suppose that we use the following activation function \begin{align} f(x)= \begin{cases} 1, \text{ if }x =1\\ 0, \text{ otherwise} \end{cases} \...
rahs's user avatar
  • 163
7 votes
1 answer
3k views

Which Rosenblatt's paper describes Rosenblatt's perceptron training algorithm?

I struggle to find Rosenblatt's perceptron training algorithm in any of his publications from 1957 - 1961, namely: Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms The ...
Cryptiex's user avatar