Questions tagged [multilayer-perceptrons]

For question about Multi Layer Perceptron model/architecture, its training and other related details and parameters associated with the model.

27 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
12
votes
0answers
217 views

What is the number of neurons required to approximate a polynomial of degree n?

I learned about the universal approximation theorem from this guide. It states that a network even with a single hidden layer can approximate any function within some bound, given a sufficient number ...
4
votes
1answer
162 views

How to deal with padded inputs in a fully connected feed forward network?

I have a fully connected network that takes in a variable-length input padded with 0. However, the network doesn't seem to be learning and I am guessing that the high number of zeros in the input ...
3
votes
2answers
132 views

Which online machine learning technique to use for multi-class classification problem with multiple inputs?

I have the following problem. We have $4$ separate discrete inputs, which can take any integer value between $-63$ and $63$. The output is also supposed to be a discrete value between $-63$ and $63$. ...
2
votes
0answers
46 views

Recent algorithms for correcting mislabeled data using multilayer perceptrons

I am doing literature research on algorithms for correcting mislabeled data using multilayer perceptrons. Found an "old" paper An algorithm for correcting mislabeled data (2001) by Xinchuan Zeng et al....
2
votes
0answers
197 views

How I can predict the next number in a sequence with a neural network?

I've been dabbling with machine learning and neural networks (namely, resnet50) for a few months now, mostly doing image recognition. I am currently trying to make a program that, given a string of ...
2
votes
2answers
441 views

Unable to overfit using MLP

I'm building a 5-class classifier with a private dataset. Each data sample has 67 features and there are about 40000 samples. Samples of a particular class were duplicated to overcome class imbalance ...
2
votes
1answer
132 views

Is there a mathematical theory behind why MLP can classify handwritten digits?

I'm trying to really understand how multi-layer perceptrons work. I want to prove mathematically that MLP's can classify handwritten digits. The only thing I really have is that each perceptron can ...
1
vote
0answers
38 views

Is the capability of RNN more than the capability of MLP?

Consider the following excerpt paragraph taken from the section titled "Recurrent Neural Networks" of the chapter 10: Sequence Modeling: Recurrent and Recursive Nets of the textbook named ...
1
vote
0answers
24 views

Does Godel's incompleteness theorems restricts the scope of connectionist-AI?

It is well-known that Godel's incompleteness theorems restricted the reachability of symbolic-AI, which is dependent on mathematical logic. But, I am wondering whether it has any impact on the ...
1
vote
0answers
50 views

Is the VC dimension of a MLP regressor a valid upper bound on how many points it can exactly fit?

I want to calculate an upper bound on how many training points an MLP regressor can fit with ~0 error. I don't care about the test error, I want to overfit as much as possible the (few) training ...
1
vote
0answers
38 views

Model unfit for some part of spiral data despite low error

I'm current testing a model for spiral data. After 500 epoches, loss is 0.04 but the result is still unmatch with some part of the training data. (bottom left) The model has 2 hidden tanh x 16 units ...
1
vote
0answers
32 views

Does a varying ANN model accuracy mean underfitting or overfitting?

Background: This is for a simulated robot with four legs, walking on a flat terrain. The ANN (an MLP) is given inputs as the robot's body angle, positions and angle of each leg with respect to the ...
1
vote
0answers
30 views

Applying Machine Learning to 2D Laser Scanner Data

We are using 2D Laser Scanner to scan various objects of different geometric shapes for e.g. cylinder, spiked, cylinder with notch, cylinder with curved edges e.t.c. The dataset contains points in the ...
1
vote
0answers
46 views

Is there data available about successful neural network architectures?

I am curious to if there is data available for MLP architectures in use today, their initial architecture, the steps that were taken to improve the architecture to an acceptable state and what the ...
1
vote
0answers
90 views

How can an ANN efficiently predict multiple numbers with fixed sum (in other words, proportions)?

I need a neural network (or any other solution) to predict 3 values which sum equals a fixed number (100). This will help me calculate proportions. Which is the most efficient way to do this? The ...
1
vote
0answers
221 views

Sigmoid output layer and Cross-Entropy cost function

I use Sigmoid activation function for neurons at output layer of my Multi-Layer Perceptron also, I use cross-entropy cost function. As I know when activation functions like Tanh is used in output ...
0
votes
1answer
13 views

Why is training all layers at a time effective for a multi-layer autoencoder?

This training of all layers of a CNN simultaneously is standard practice today. It is found in every CNN (AlexNet (2012), VGG, Inception, GANs, etc) and even pre-CNN networks such as Le et al. 2012. ...
0
votes
0answers
27 views

Which one is better for interpolation and extrapolation: the multi-layer perceptron or radial basis function network?

Artificial neural networks (ANNs) can be used for function estimation, furthermore, in order to estimate a function there are two techniques: interpolation and extrapolation Between the multilayer ...
0
votes
0answers
20 views

Rank of gradient-of-loss with respect to layer weights in an MLP

The paper: https://arxiv.org/abs/2110.11309, makes the following claim at the end of page 3: The gradient of loss $L$ with respect to weights $W_l$ of an MLP is a rank-1 matrix for each of B batch ...
0
votes
0answers
6 views

The ratio between number of units in multi-input model

I have the model that accepts two inputs: Image from camera Speed of the car I can create some CNN layers to process the image input and some MLP layers to process other type of data (for example ...
0
votes
0answers
19 views

What's the effect of increasing hidden nodes?

Topic Demarcation I find many topics on "how to choose the number of hidden nodes". I'm not interested in the answer to that question. What I learned I learned, that if you increase the ...
0
votes
0answers
26 views

Keras MLP performing better than Transformers

I'm working on a comparative study using some models in a sentiment analysis task: MLPs and LSTMs with and without the use of word embeddings (GloVe and Word2Vec) and two Transformer-based models (...
0
votes
0answers
23 views

GAN - relation of input size and hidden layer size

I'm adapting a GAN described here used for generating binary output. It's trained on binarized MNIST data, with a size of 28x28 so 784 values. I want to adapt it to train on and generate 1D vectors ...
0
votes
0answers
38 views

Why am I not getting a good accuracy but bad precision and recall for this binary classification problem (with an unbalanced dataset)?

I'm working on a simple MLP classification problem where I need to have the output layer as softmax. The dataset that I've been using needs to pass through a parse that I made to remove NaN and change ...
0
votes
0answers
47 views

What are the math theorems regarding the Multilayer Perceptron?

I've come across a theorem "Convergence theorem Simple Perceptron" for the first time, here-> https://zaguan.unizar.es/record/69205/files/TAZ-TFG-2018-148.pdf, page 27, (is in Spanish) ...
0
votes
0answers
119 views

What is the difference between the forward pass of the Multi-Layer Perceptron, Deep AutoEncoder and Deep Belief Network?

Multi-Layer Perceptron (MLP), Deep AutoEncoder (DAE), and Deep Belief Network (DBN) are trained differently. However, do they follow the same process during the inference phase, i.e., do they ...
0
votes
0answers
35 views

How to draw a 3-dimensonal shape's neural network

I am reading an exam question about NN (that I cannot publish, for copyright reasons). The question says: 'Construct a rectangle in 2D space. Define the lines, and then define the weights and ...