Questions tagged [multilayer-perceptrons]
For question about Multi Layer Perceptron model/architecture, its training and other related details and parameters associated with the model.
67
questions
0
votes
1
answer
29
views
What are all the possible usages of 'multilayer perceptron'?
The term 'multilayer perceptron' has been used in literature in various ways in the literature.
I am presenting some of them below
As a feed-forward neural network [1].
As a fully connected feed-...
- 3,109
2
votes
1
answer
42
views
How to decode P bits that represent a random weight generator?
So I've been tasked by my neural network professor at university to replicate the following research: Intelligent Breast Cancer Diagnosis Using Hybrid GA-ANN.
Each chromosome represents a possible net,...
- 21
0
votes
1
answer
14
views
Do Quo et al (2013) perform backpropagation between layers?
Le et al. 2013's non-weight sharing CNN has inspired me to ask two questions on this site previously.
When training the three-layer autoencoder, do they compute dL/dW (where L is equation 1) ...
- 151
0
votes
1
answer
85
views
How to make a proper approximation of Sine function with Neural Networks?
TL;DR;
How to build a neural network that properly approximates the sine function with different ranges?
Context and Question:
From this question I decided to use the Sergey's answer, however I used a ...
- 111
1
vote
1
answer
30
views
Does a second-order fully-connected layer have any uses?
I was thinking about implementing second-order regression via a fully-connected layer, and I came up with this:
$X$ is the input data, shaped $(features, batch\_number)$.
$w0$ is the bias, shaped $(...
- 113
1
vote
1
answer
34
views
Why is training all layers at a time effective for a multi-layer autoencoder?
This training of all layers of a CNN simultaneously is standard practice today. It is found in every CNN (AlexNet (2012), VGG, Inception, GANs, etc) and even pre-CNN networks such as Le et al. 2012.
...
- 151
1
vote
0
answers
52
views
Is the capability of RNN more than the capability of MLP?
Consider the following excerpt paragraph taken from the section titled "Recurrent Neural Networks" of the chapter 10: Sequence Modeling: Recurrent and Recursive Nets of the textbook named ...
- 3,109
1
vote
0
answers
29
views
Does Godel's incompleteness theorems restricts the scope of connectionist-AI?
It is well-known that Godel's incompleteness theorems restricted the reachability of symbolic-AI, which is dependent on mathematical logic.
But, I am wondering whether it has any impact on the ...
- 3,109
0
votes
0
answers
36
views
Which one is better for interpolation and extrapolation: the multi-layer perceptron or radial basis function network?
Artificial neural networks (ANNs) can be used for function estimation, furthermore, in order to estimate a function there are two techniques:
interpolation
and
extrapolation
Between the multilayer ...
- 101
1
vote
1
answer
54
views
Are the capabilities of connectionist AI and symbolic AI the same?
The universal approximation theorem says that MLP with a single hidden layer and enough number of neurons can able to approximate any bounded continuous function. You can validate it from the ...
- 3,109
1
vote
1
answer
34
views
Rank of gradient-of-loss with respect to layer weights in an MLP
The paper: https://arxiv.org/abs/2110.11309, makes the following claim at the end of page 3:
The gradient of loss $L$ with respect to weights $W_l$ of an MLP is a rank-1 matrix for each of B batch ...
- 63
1
vote
0
answers
55
views
Is the VC dimension of a MLP regressor a valid upper bound on how many points it can exactly fit?
I want to calculate an upper bound on how many training points an MLP regressor can fit with ~0 error. I don't care about the test error, I want to overfit as much as possible the (few) training ...
- 11
1
vote
1
answer
70
views
When should we use CNN instead of MLP?
Is CNN only applicable to time-series data or image data?
When should we use CNN instead of MLP?
- 101
0
votes
1
answer
352
views
What, exactly, do mlp(64,64) and mlp(64,128,1024) mean in PointNet, and how many input neurons does 1 (x,y,z) point have?
I couldn't find out how to interpret the multilayer perceptron notation given in PointNet. Specifically, I am looking to find out what the numbers inside the parentheses of ...
- 31
0
votes
0
answers
6
views
The ratio between number of units in multi-input model
I have the model that accepts two inputs:
Image from camera
Speed of the car
I can create some CNN layers to process the image input and some MLP layers to process other type of data (for example ...
- 183
0
votes
0
answers
20
views
What's the effect of increasing hidden nodes?
Topic Demarcation
I find many topics on "how to choose the number of hidden nodes". I'm not interested in the answer to that question.
What I learned
I learned, that if you increase the ...
- 101
1
vote
1
answer
71
views
Why doesn't the LSTM model improve the time-series forecasting significantly with respect to the MLP model?
I have recently started learning time series forecasting. I have a dataset of the weekly payment history of 10k clients over 1 year, and I want to predict the future 5 payments for a test set of 1k ...
- 111
0
votes
0
answers
34
views
Keras MLP performing better than Transformers
I'm working on a comparative study using some models in a sentiment analysis task: MLPs and LSTMs with and without the use of word embeddings (GloVe and Word2Vec) and two Transformer-based models (...
0
votes
1
answer
67
views
About the choice of the activation functions in the Multilayer Perceptron, and on what does this depends?
I've read in this: F. Rosenblatt, Principles of neurodynamics. perceptrons and the theory of brain mechanisms that in the Multilayer Perceptron the activation functions in the second, third, ..., are ...
0
votes
0
answers
49
views
What are the math theorems regarding the Multilayer Perceptron?
I've come across a theorem "Convergence theorem
Simple Perceptron" for the first time, here-> https://zaguan.unizar.es/record/69205/files/TAZ-TFG-2018-148.pdf, page 27, (is in Spanish)
...
2
votes
1
answer
233
views
Why is the backpropagation algorithm used to train the multilayer perceptron?
I've read in the book Neural Network Design, by Martin Hagan et al. (chapter 11), that, to train the feed-forward neural network (aka multilayer perceptron), one uses the backpropagation algorithm.
...
0
votes
0
answers
128
views
What is the difference between the forward pass of the Multi-Layer Perceptron, Deep AutoEncoder and Deep Belief Network?
Multi-Layer Perceptron (MLP), Deep AutoEncoder (DAE), and Deep Belief Network (DBN) are trained differently.
However, do they follow the same process during the inference phase, i.e., do they ...
- 73
1
vote
1
answer
65
views
Can RNNs get inputs and produce outputs similar to the inputs and outputs of FFNNs?
RNN and LSTM models have many architectures that can be modified. We can also compose their input and output data. However, in the examples that I found on the web, the inputs and outputs of RNNs/...
- 45
1
vote
1
answer
134
views
What are examples of good free books that cover the back-propagation algorithm?
What are examples of good free books that cover the back-propagation used to train multilayer perceptrons? I've just started to learn about artificial neural networks, so I'm looking for books that ...
5
votes
1
answer
565
views
Why do feedforward neural networks require the inputs to be of a fixed size, while RNNs can process variable-size inputs?
Why does a vanilla feedforward neural network only accept a fixed input size, while RNNs are capable of taking a series of inputs with no predetermined limit on the size? Can anyone elaborate on this ...
- 53
12
votes
0
answers
280
views
What is the number of neurons required to approximate a polynomial of degree n?
I learned about the universal approximation theorem from this guide. It states that a network even with a single hidden layer can approximate any function within some bound, given a sufficient number ...
- 663
0
votes
0
answers
43
views
How to draw a 3-dimensonal shape's neural network
I am reading an exam question about NN (that I cannot publish, for copyright reasons). The question says: 'Construct a rectangle in 2D space. Define the lines, and then define the weights and ...
- 267
1
vote
2
answers
365
views
Why don't neural networks project the data into higher dimensions first, then reduce the size of each layer thereafter?
Background
From my understanding (and following along with this blog post), (deep) neural networks apply transformations to the data such that the data's representation to the next layer (or ...
- 123
0
votes
1
answer
451
views
Keras DQN Model with Multiple Inputs and Multiple Outputs [closed]
I am trying to create a DQN agent where I have 2 inputs: the agent's position and a matrix of 0s and 1s.
The output is composed of the agent's new chosen position, a matrix of 0s and 1s (different ...
- 146
3
votes
1
answer
71
views
Is there a common way to build a neural network that seeks to extract spatial and temporal information simultaneously?
Is there a common way to build a neural network that seeks to extract spatial and temporal information simultaneously? Is there an agreed up protocol on how to extract this information?
What ...
- 165
1
vote
2
answers
392
views
How is the error calculated with multiple output neurons in the neural network?
Machine Learning books generally explains that the error calculated for a given sample $i$ is:
$e_i = y_i - \hat{y_i}$
Where $\hat{y}$ is the target output and $y$ is the actual output given by the ...
- 111
3
votes
1
answer
426
views
Why does every neuron in hidden layers of a multi-layer perceptron typically have the same activation function? [duplicate]
Why does every neuron in a hidden layer of a multi-layer perceptron (MLP) typically have the same activation function as every other neuron in the same or other hidden layers (so I exclude the output ...
- 597
0
votes
1
answer
1k
views
Why can't MLPs perform non-linear regression and classification?
In this page it's told:
In Single Perceptron / Multi-layer Perceptron(MLP), we only have linear separability because they are composed of input and output layers(some hidden layers in MLP)
What ...
- 157
2
votes
2
answers
113
views
What are standard datasets for fully connected neural networks?
I am looking for datasets that are used as a testing standard in the fully connected neural networks (FCNN). For example, in the image recognition and CNN, CIFAR datasets are used in most of the ...
- 313
3
votes
2
answers
581
views
Why is it called back-propagation?
While looking at the mathematics of the back-propagation algorithm for a multi-layer perceptron, I noticed that in order to find the partial derivative of the cost function with respect to a weight (...
- 133
5
votes
2
answers
612
views
Can neurons in MLP and filters in CNN be compared?
I know they are not the same in working, but an input layer sends the input to $n$ neurons with a set of weights, based on these weights and the activation layer, it produces an output that can be fed ...
- 183
4
votes
1
answer
72
views
Why MLP cannot approximate a closed shape function?
[TL;DR]
I generated two classes Red and Blue on a 2D space. Red are points on Unit Circle and Blue are points on a Circle Ring with radius limits (3,4). I tried to train a Multi Layer Perceptron ...
- 143
2
votes
0
answers
48
views
Recent algorithms for correcting mislabeled data using multilayer perceptrons
I am doing literature research on algorithms for correcting mislabeled data using multilayer perceptrons. Found an "old" paper An algorithm for correcting mislabeled data (2001) by Xinchuan Zeng et al....
- 63
4
votes
1
answer
103
views
Are the labels updated during training in the algorithm presented in "An algorithm for correcting mislabeled data"?
I am trying to understand an algorithm for correcting mislabeled data in the paper An algorithm for correcting mislabeled data (2001) by Xinchuan Zeng et al. The authors are suggesting to update the ...
- 63
1
vote
0
answers
43
views
Model unfit for some part of spiral data despite low error
I'm current testing a model for spiral data. After 500 epoches, loss is 0.04 but the result is still unmatch with some part of the training data. (bottom left)
The model has 2 hidden tanh x 16 units ...
- 11
2
votes
1
answer
136
views
Is there a mathematical theory behind why MLP can classify handwritten digits?
I'm trying to really understand how multi-layer perceptrons work. I want to prove mathematically that MLP's can classify handwritten digits. The only thing I really have is that each perceptron can ...
- 597
2
votes
0
answers
284
views
How I can predict the next number in a sequence with a neural network?
I've been dabbling with machine learning and neural networks (namely, resnet50) for a few months now, mostly doing image recognition. I am currently trying to make a program that, given a string of ...
- 21
1
vote
1
answer
96
views
Why does my model overfit on pseudo-random numbers training data?
I am trying to predict pseudo-random numbers using the past numbers with a multiplayer perceptron. The error while training is very low. However, as soon as I test it with a test set, the model ...
- 133
4
votes
1
answer
96
views
Is it expected that adding an additional hidden layer to my 3-layer ANN reduces accuracy significantly?
I've been using several resources to implement my own artificial neural network package in C++.
Among some of the resources I've been using are
https://www.anotsorandomwalk.com/backpropagation-...
- 25
1
vote
0
answers
34
views
Does a varying ANN model accuracy mean underfitting or overfitting?
Background:
This is for a simulated robot with four legs, walking on a flat terrain. The ANN (an MLP) is given inputs as the robot's body angle, positions and angle of each leg with respect to the ...
- 111
1
vote
0
answers
32
views
Applying Machine Learning to 2D Laser Scanner Data
We are using 2D Laser Scanner to scan various objects of different geometric shapes for e.g. cylinder, spiked, cylinder with notch, cylinder with curved edges e.t.c. The dataset contains points in the ...
- 11
8
votes
2
answers
2k
views
Why use a recurrent neural network over a feedforward neural network for sequence prediction?
If recurrent neural networks (RNNs) are used to capture prior information, couldn't the same thing be achieved by a feedforward neural network (FFNN) or multi-layer perceptron (MLP) where the inputs ...
- 263
0
votes
1
answer
111
views
How can we print weights per iteration in a simple feed forward MLP for an specific class?
im working on a project in which I have to make a multi-layer perceptron with two hidden layers with 3 nodes in each. The target value in my data contains 8 unique values/classes. One of the tasks ...
1
vote
0
answers
46
views
Is there data available about successful neural network architectures?
I am curious to if there is data available for MLP architectures in use today, their initial architecture, the steps that were taken to improve the architecture to an acceptable state and what the ...
- 243
2
votes
3
answers
427
views
One vs multiple output neurons
Consider an MLP that outputs an integer 'rating' of 0 to 4. Would it be correct to say this could be modeled in either of the following ways:
map each rating in the dataset to a 'normalized set' ...
- 59