Questions tagged [multilayer-perceptrons]
For question about Multi Layer Perceptron model/architecture, its training and other related details and parameters associated with the model.
25
questions with no upvoted or accepted answers
17
votes
0
answers
579
views
What is the number of neurons required to approximate a polynomial of degree n?
I learned about the universal approximation theorem from this guide. It states that a network even with a single hidden layer can approximate any function within some bound, given a sufficient number ...
- 723
3
votes
0
answers
44
views
Why does a neural network struggle to solve this simple problem?
Consider the following problem:
Given a vector x of size dim with values between 0 and 1 (exclusive), determine if ...
- 191
3
votes
2
answers
192
views
Which online machine learning technique to use for multi-class classification problem with multiple inputs?
I have the following problem. We have $4$ separate discrete inputs, which can take any integer value between $-63$ and $63$. The output is also supposed to be a discrete value between $-63$ and $63$. ...
- 31
2
votes
1
answer
100
views
Multi-objective training involving maximization of one loss function and minimization of another
I need my model to predict $s$ from my data $x$. Additionally, I need the model to not use signals in $x$ that are predictive of a separate target $a$. My approach is to transform $x$ into a ...
- 121
2
votes
1
answer
50
views
How to decode P bits that represent a random weight generator?
So I've been tasked by my neural network professor at university to replicate the following research: Intelligent Breast Cancer Diagnosis Using Hybrid GA-ANN.
Each chromosome represents a possible net,...
- 21
2
votes
0
answers
557
views
How I can predict the next number in a sequence with a neural network?
I've been dabbling with machine learning and neural networks (namely, resnet50) for a few months now, mostly doing image recognition. I am currently trying to make a program that, given a string of ...
- 21
2
votes
2
answers
540
views
Unable to overfit using MLP
I'm building a 5-class classifier with a private dataset. Each data sample has 67 features and there are about 40000 samples. Samples of a particular class were duplicated to overcome class imbalance ...
2
votes
1
answer
141
views
Is there a mathematical theory behind why MLP can classify handwritten digits?
I'm trying to really understand how multi-layer perceptrons work. I want to prove mathematically that MLP's can classify handwritten digits. The only thing I really have is that each perceptron can ...
- 687
1
vote
1
answer
52
views
Why is training all layers at a time effective for a multi-layer autoencoder?
This training of all layers of a CNN simultaneously is standard practice today. It is found in every CNN (AlexNet (2012), VGG, Inception, GANs, etc) and even pre-CNN networks such as Le et al. 2012.
...
- 151
1
vote
0
answers
59
views
Is the capability of RNN more than the capability of MLP?
Consider the following excerpt paragraph taken from the section titled "Recurrent Neural Networks" of the chapter 10: Sequence Modeling: Recurrent and Recursive Nets of the textbook named ...
- 3,481
1
vote
0
answers
47
views
Does Godel's incompleteness theorems restricts the scope of connectionist-AI?
It is well-known that Godel's incompleteness theorems restricted the reachability of symbolic-AI, which is dependent on mathematical logic.
But, I am wondering whether it has any impact on the ...
- 3,481
1
vote
1
answer
58
views
Rank of gradient-of-loss with respect to layer weights in an MLP
The paper: https://arxiv.org/abs/2110.11309, makes the following claim at the end of page 3:
The gradient of loss $L$ with respect to weights $W_l$ of an MLP is a rank-1 matrix for each of B batch ...
- 63
1
vote
0
answers
63
views
Is the VC dimension of a MLP regressor a valid upper bound on how many points it can exactly fit?
I want to calculate an upper bound on how many training points an MLP regressor can fit with ~0 error. I don't care about the test error, I want to overfit as much as possible the (few) training ...
- 11
1
vote
0
answers
59
views
Model unfit for some part of spiral data despite low error
I'm current testing a model for spiral data. After 500 epoches, loss is 0.04 but the result is still unmatch with some part of the training data. (bottom left)
(source: upsieutoc.com)
The model has 2 ...
- 11
1
vote
0
answers
43
views
Does a varying ANN model accuracy mean underfitting or overfitting?
Background:
This is for a simulated robot with four legs, walking on a flat terrain. The ANN (an MLP) is given inputs as the robot's body angle, positions and angle of each leg with respect to the ...
- 111
1
vote
0
answers
50
views
Applying Machine Learning to 2D Laser Scanner Data
We are using 2D Laser Scanner to scan various objects of different geometric shapes for e.g. cylinder, spiked, cylinder with notch, cylinder with curved edges e.t.c. The dataset contains points in the ...
- 11
1
vote
0
answers
46
views
Is there data available about successful neural network architectures?
I am curious to if there is data available for MLP architectures in use today, their initial architecture, the steps that were taken to improve the architecture to an acceptable state and what the ...
- 253
1
vote
0
answers
141
views
How can an ANN efficiently predict multiple numbers with fixed sum (in other words, proportions)?
I need a neural network (or any other solution) to predict 3 values which sum equals a fixed number (100). This will help me calculate proportions. Which is the most efficient way to do this?
The ...
- 151
1
vote
0
answers
237
views
Sigmoid output layer and Cross-Entropy cost function
I use Sigmoid activation function for neurons at output layer of my Multi-Layer Perceptron also, I use cross-entropy cost function. As I know when activation functions like Tanh is used in output ...
- 185
0
votes
0
answers
39
views
Are autoencoders computationally cheaper than MLPs with the same number of neurons?
Are autoencoders computationally cheaper than other neural networks such as MLP with the same number of neurons?
I have read in some papers that autoencoders train the network faster, and I could ...
- 1
0
votes
0
answers
33
views
What is the right way to normalize the initial weights of a fully connected layer using a SiLu (Sigmoid-weighted Linear Unit) activation function?
I've been writing a deep learning Java framework as a way for myself to learn how it all works and I have had a decent amount of success so far. Best performance is just over 90% accuracy with three ...
0
votes
1
answer
23
views
Do Quo et al (2013) perform backpropagation between layers?
Le et al. 2013's non-weight sharing CNN has inspired me to ask two questions on this site previously.
When training the three-layer autoencoder, do they compute dL/dW (where L is equation 1) ...
- 151
0
votes
0
answers
57
views
What are the math theorems regarding the Multilayer Perceptron?
I've come across a theorem "Convergence theorem
Simple Perceptron" for the first time, here-> https://zaguan.unizar.es/record/69205/files/TAZ-TFG-2018-148.pdf, page 27, (is in Spanish)
...
0
votes
0
answers
148
views
What is the difference between the forward pass of the Multi-Layer Perceptron, Deep AutoEncoder and Deep Belief Network?
Multi-Layer Perceptron (MLP), Deep AutoEncoder (DAE), and Deep Belief Network (DBN) are trained differently.
However, do they follow the same process during the inference phase, i.e., do they ...
- 73
0
votes
0
answers
60
views
How to draw a 3-dimensonal shape's neural network
I am reading an exam question about NN (that I cannot publish, for copyright reasons). The question says: 'Construct a rectangle in 2D space. Define the lines, and then define the weights and ...
- 277