Skip to main content
14 votes

Did Minsky and Papert know that multi-layer perceptrons could solve XOR?

Whether Minsky knew or not, it was definitely known to Rosenblatt, as he published those results in his really pioneering report - Principles of Neurodynamics: Perceptrons and the Theory of Brain ...
Cel's user avatar
  • 141
13 votes

Did Minsky and Papert know that multi-layer perceptrons could solve XOR?

There does not appear to be a historical consensus on this. The Wikipedia page on the Perceptrons book (which does not come down on either side) gives an argument that the ability of MLPs to compute ...
NietzscheanAI's user avatar
8 votes

Why is it called back-propagation?

Why is it called back-propagation? I don't think there is anything special here! It's called back-propagation (BP) because, after the forward pass, you compute the partial derivative of the loss ...
nbro's user avatar
  • 40.9k
7 votes

Why use a recurrent neural network over a feedforward neural network for sequence prediction?

An RNN or LSTM have the advantage of "remembering" the past inputs, to improve performance over prediction of a time-series data. If you use a neural network over like the past 500 characters, this ...
Clement's user avatar
  • 1,745
7 votes

Did Minsky and Papert know that multi-layer perceptrons could solve XOR?

In section 13.2 Other Multilayer Machines (pp. 231-232) of the book Perceptrons: An Introduction to Computational Geometry (expanded edition, third printing, 1988) Minsky and Papert actually talk ...
nbro's user avatar
  • 40.9k
7 votes
Accepted

Why do feedforward neural networks require the inputs to be of a fixed size, while RNNs can process variable-size inputs?

You are talking about two different types of 'size'. The size of the input for a FFNN and a RNN must always remain fixed for the same network architecture, i.e. they take in a vector $x \in \mathbb{R}^...
David's user avatar
  • 4,920
5 votes

Is a multilayer perceptron a recursive function?

Inherently, no. The MLP is just a data structure. It represents a function, but a standard MLP is just representing an input-output mapping, and there's no recursive structure to it. On the other ...
John Doucette's user avatar
4 votes

What are some datasets to train an MLP on simple tasks?

There are a ton of sample datasets our there you can play with. A bunch of good ones install with R in the datasets package. Luckily you can download them independently if you're not an R user. Try ...
mindcrime's user avatar
  • 3,767
4 votes
Accepted

Can neurons in MLP and filters in CNN be compared?

tl;dr The equivalent to a neuron in a Fully-Connected (FC) layer is the kernel (or filter) of a Convolution layer Differences The neurons of these two types of layers have two key differences. These ...
Djib2011's user avatar
  • 3,193
4 votes

Why is the backpropagation algorithm used to train the multilayer perceptron?

According to wikipedia of backpropagation: In fitting a neural network, backpropagation computes the gradient of the loss function during supervised learning with respect to the weights of the ...
cinch's user avatar
  • 2,277
4 votes
Accepted

Why is automatic differentiation still used, if today's computers can calculate symbolic derivatives quite fast?

That depends on the point-of-view. If you have some function given as a string of its mathematical formula and the output is the fully inserted string of its derivative, then the chain rule $(u\circ v)...
Lutz Lehmann's user avatar
3 votes
Accepted

Why is it called back-propagation?

Have a look at the following article Principles of training multi-layer neural network using backpropagation. It was very useful to me. You can also see here an example of backpropagation in Matlab. ...
david david's user avatar
3 votes
Accepted

Why MLP cannot approximate a closed shape function?

In neural networks, the family of functions and the shapes that they can make for decision surfaces is determined by the activation function you use (in your case, ...
Neil Slater's user avatar
  • 32.7k
3 votes
Accepted

Why use a recurrent neural network over a feedforward neural network for sequence prediction?

Assumptions Different model structures encode different assumptions - while we often make simplifying assumptions that aren't strictly correct, some assumptions are more wrong than others. For ...
Peteris's user avatar
  • 883
3 votes
Accepted

How can a neural network learn to play sudoku?

I think it is the wrong way to frame sudoku as a regression problem in neural networks. Firstly, you have to understand what regression is. "Regression" is when you predict a value given certain ...
thecomplexitytheorist's user avatar
3 votes

Is a multilayer perceptron a recursive function?

Sure, you can define plenty of things we don't generally need to regard as recursive as so. An MLP is just a series of functions applied to its input. This can be loosely formulated as $$ o_n = f(o_{...
Daniel's user avatar
  • 326
3 votes
Accepted

Is it a valid assumption that a purely MLP based tic-tac-toe player will learn lookahead strategies?

A MLP only does pattern recognition, it will not learn search. Tictactoe, (Oughts and Crosses), is such a simple game that your network should learn the moves from the training data by heart, no ...
BlindKungFuMaster's user avatar
3 votes
Accepted

Is there a common way to build a neural network that seeks to extract spatial and temporal information simultaneously?

Yes, there are different ways. What I think you are looking for is under the research field of Localization and Mapping. Which divides in the following subfields: For getting current (the robot) ...
JVGD's user avatar
  • 1,138
3 votes
Accepted

What, exactly, do mlp(64,64) and mlp(64,128,1024) mean in PointNet, and how many input neurons does 1 (x,y,z) point have?

The following link satisfied my inquiries: https://www.mdpi.com/1999-4907/12/2/131/htm I hope this is useful for someone else! Justin
Justin's user avatar
  • 41
3 votes
Accepted

When should we use CNN instead of MLP?

CNN applies the same filters to every "chunk" of the input data. It's applicable when you think every chunk should be processed the same way. For example, we think a face in the top-left of ...
Quitting Due To Antisemitism's user avatar
3 votes

Why is a simple regression problem so hard for an MLP to learn?

An interesting problem. This network has only 933 trainable parameters, and obtains MeanAbsolutePercentageError of 0.01 - 0.04. It is based on a softmax activation, ...
NikoNyrh's user avatar
  • 777
3 votes
Accepted

Why does averaging attention-weighted positions reduce the effective resolution in transformers?

I think that the idea of "reduced effective resolution" from averaging is best understood through seeing how the proposed multi-head attention architecture fixes the issue. Specifically, ...
Andrew Du's user avatar
3 votes
Accepted

Is it a requirement/recommendation to normalize my inputs into [0,1] range?

Generally, between -1 and 1 are ideal, though you can get away with a wider range. For example, using the z-score as the range, you will be outside of this range, sometimes by quite a bit (say, -30 ...
David Hoelzer's user avatar
2 votes

What are some datasets to train an MLP on simple tasks?

A popular dataset is the fisher iris dataset. It consists of 150 samples each with a dimensionality of 4. You can find it at http://archive.ics.uci.edu/ml/datasets/Iris
hh32's user avatar
  • 175
2 votes

Using a MLP to predict a 12x12 matrix

The short answer to your question is: you probably do not fully know your data. remember that ML is no magic wand. It needs your understanding of the data and the behavior of it. Although it is ...
Alireza's user avatar
  • 405
2 votes

Comments on my proposed "Jitter" neuron

Well, adding gaussian noise is a very common regularisation method. Maybe this paper is interesting to you. They also have very small datasets. In the end there is only so much you can get out of a ...
BlindKungFuMaster's user avatar
2 votes

How can a neural network learn to play sudoku?

You can take a look at this paper that solving your problem with a neural network. You can use the pytorch implementation of the satnet layer : satnet layer API. In this supervised setup the layer ...
Adrien Forbu's user avatar
2 votes
Accepted

Is it expected that adding an additional hidden layer to my 3-layer ANN reduces accuracy significantly?

You probably got the back propagation wrong. I have done a test on the accuracy on adding an extra layer and the accuracy went up from 94% to 96% for me. See this for details: https://colab.research....
Clement's user avatar
  • 1,745
2 votes
Accepted

Are the labels updated during training in the algorithm presented in "An algorithm for correcting mislabeled data"?

I think that making some draws might help. Below I tried to draw the model architecture. We start with classic feed-forward structure: input represented by a vector I with length f (number of ...
Edoardo Guerriero's user avatar
2 votes
Accepted

Why can't MLPs perform non-linear regression and classification?

In Single Perceptron / Multi-layer Perceptron(MLP), we only have linear separability because they are composed of input and output layers(some hidden layers in MLP) This is wrong. A multi-layer ...
nbro's user avatar
  • 40.9k

Only top scored, non community-wiki answers of a minimum length are eligible