Questions tagged [multilayer-perceptrons]

For question about Multi Layer Perceptron model/architecture, its training and other related details and parameters associated with the model.

24 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
15 votes
0 answers
407 views

What is the number of neurons required to approximate a polynomial of degree n?

I learned about the universal approximation theorem from this guide. It states that a network even with a single hidden layer can approximate any function within some bound, given a sufficient number ...
  • 693
3 votes
2 answers
166 views

Which online machine learning technique to use for multi-class classification problem with multiple inputs?

I have the following problem. We have $4$ separate discrete inputs, which can take any integer value between $-63$ and $63$. The output is also supposed to be a discrete value between $-63$ and $63$. ...
2 votes
1 answer
77 views

Multi-objective training involving maximization of one loss function and minimization of another

I need my model to predict $s$ from my data $x$. Additionally, I need the model to not use signals in $x$ that are predictive of a separate target $a$. My approach is to transform $x$ into a ...
2 votes
1 answer
46 views

How to decode P bits that represent a random weight generator?

So I've been tasked by my neural network professor at university to replicate the following research: Intelligent Breast Cancer Diagnosis Using Hybrid GA-ANN. Each chromosome represents a possible net,...
2 votes
0 answers
478 views

How I can predict the next number in a sequence with a neural network?

I've been dabbling with machine learning and neural networks (namely, resnet50) for a few months now, mostly doing image recognition. I am currently trying to make a program that, given a string of ...
2 votes
2 answers
521 views

Unable to overfit using MLP

I'm building a 5-class classifier with a private dataset. Each data sample has 67 features and there are about 40000 samples. Samples of a particular class were duplicated to overcome class imbalance ...
2 votes
1 answer
140 views

Is there a mathematical theory behind why MLP can classify handwritten digits?

I'm trying to really understand how multi-layer perceptrons work. I want to prove mathematically that MLP's can classify handwritten digits. The only thing I really have is that each perceptron can ...
1 vote
1 answer
39 views

Why is training all layers at a time effective for a multi-layer autoencoder?

This training of all layers of a CNN simultaneously is standard practice today. It is found in every CNN (AlexNet (2012), VGG, Inception, GANs, etc) and even pre-CNN networks such as Le et al. 2012. ...
1 vote
0 answers
57 views

Is the capability of RNN more than the capability of MLP?

Consider the following excerpt paragraph taken from the section titled "Recurrent Neural Networks" of the chapter 10: Sequence Modeling: Recurrent and Recursive Nets of the textbook named ...
  • 3,371
1 vote
0 answers
40 views

Does Godel's incompleteness theorems restricts the scope of connectionist-AI?

It is well-known that Godel's incompleteness theorems restricted the reachability of symbolic-AI, which is dependent on mathematical logic. But, I am wondering whether it has any impact on the ...
  • 3,371
1 vote
1 answer
53 views

Rank of gradient-of-loss with respect to layer weights in an MLP

The paper: https://arxiv.org/abs/2110.11309, makes the following claim at the end of page 3: The gradient of loss $L$ with respect to weights $W_l$ of an MLP is a rank-1 matrix for each of B batch ...
  • 63
1 vote
0 answers
58 views

Is the VC dimension of a MLP regressor a valid upper bound on how many points it can exactly fit?

I want to calculate an upper bound on how many training points an MLP regressor can fit with ~0 error. I don't care about the test error, I want to overfit as much as possible the (few) training ...
1 vote
0 answers
49 views

Model unfit for some part of spiral data despite low error

I'm current testing a model for spiral data. After 500 epoches, loss is 0.04 but the result is still unmatch with some part of the training data. (bottom left) The model has 2 hidden tanh x 16 units ...
  • 11
1 vote
0 answers
40 views

Does a varying ANN model accuracy mean underfitting or overfitting?

Background: This is for a simulated robot with four legs, walking on a flat terrain. The ANN (an MLP) is given inputs as the robot's body angle, positions and angle of each leg with respect to the ...
1 vote
0 answers
39 views

Applying Machine Learning to 2D Laser Scanner Data

We are using 2D Laser Scanner to scan various objects of different geometric shapes for e.g. cylinder, spiked, cylinder with notch, cylinder with curved edges e.t.c. The dataset contains points in the ...
  • 11
1 vote
0 answers
46 views

Is there data available about successful neural network architectures?

I am curious to if there is data available for MLP architectures in use today, their initial architecture, the steps that were taken to improve the architecture to an acceptable state and what the ...
  • 253
1 vote
0 answers
120 views

How can an ANN efficiently predict multiple numbers with fixed sum (in other words, proportions)?

I need a neural network (or any other solution) to predict 3 values which sum equals a fixed number (100). This will help me calculate proportions. Which is the most efficient way to do this? The ...
1 vote
0 answers
234 views

Sigmoid output layer and Cross-Entropy cost function

I use Sigmoid activation function for neurons at output layer of my Multi-Layer Perceptron also, I use cross-entropy cost function. As I know when activation functions like Tanh is used in output ...
  • 185
0 votes
0 answers
15 views

What is the right way to normalize the initial weights of a fully connected layer using a SiLu (Sigmoid-weighted Linear Unit) activation function?

I've been writing a deep learning Java framework as a way for myself to learn how it all works and I have had a decent amount of success so far. Best performance is just over 90% accuracy with three ...
0 votes
1 answer
20 views

Do Quo et al (2013) perform backpropagation between layers?

Le et al. 2013's non-weight sharing CNN has inspired me to ask two questions on this site previously. When training the three-layer autoencoder, do they compute dL/dW (where L is equation 1) ...
0 votes
0 answers
44 views

Which one is better for interpolation and extrapolation: the multi-layer perceptron or radial basis function network?

Artificial neural networks (ANNs) can be used for function estimation, furthermore, in order to estimate a function there are two techniques: interpolation and extrapolation Between the multilayer ...
  • 101
0 votes
0 answers
53 views

What are the math theorems regarding the Multilayer Perceptron?

I've come across a theorem "Convergence theorem Simple Perceptron" for the first time, here-> https://zaguan.unizar.es/record/69205/files/TAZ-TFG-2018-148.pdf, page 27, (is in Spanish) ...
0 votes
0 answers
135 views

What is the difference between the forward pass of the Multi-Layer Perceptron, Deep AutoEncoder and Deep Belief Network?

Multi-Layer Perceptron (MLP), Deep AutoEncoder (DAE), and Deep Belief Network (DBN) are trained differently. However, do they follow the same process during the inference phase, i.e., do they ...
  • 73
0 votes
0 answers
53 views

How to draw a 3-dimensonal shape's neural network

I am reading an exam question about NN (that I cannot publish, for copyright reasons). The question says: 'Construct a rectangle in 2D space. Define the lines, and then define the weights and ...