Questions tagged [multilayer-perceptrons]

For question about Multi Layer Perceptron model/architecture, its training and other related details and parameters associated with the model.

Filter by
Sorted by
Tagged with
0
votes
0answers
17 views

Rank of gradient-of-loss with respect to layer weights in an MLP

The paper: https://arxiv.org/abs/2110.11309, makes the following claim at the end of page 3: The gradient of loss $L$ with respect to weights $W_l$ of an MLP is a rank-1 matrix for each of B batch ...
1
vote
0answers
46 views

Is the VC dimension of a MLP regressor a valid upper bound on how many points it can exactly fit?

I want to calculate an upper bound on how many training points an MLP regressor can fit with ~0 error. I don't care about the test error, I want to overfit as much as possible the (few) training ...
0
votes
0answers
20 views

Why can't perceptrons approximate any function, in contrast to multilayer perceptrons? [duplicate]

What is the formal explanation and proof that multilayer perceptrons are universal approximators, as opposed to the single layer perceptron. This intutively makes sense to me, I would just like the ...
1
vote
1answer
52 views

When should we use CNN instead of MLP?

Is CNN only applicable to time-series data or image data? When should we use CNN instead of MLP?
0
votes
0answers
12 views

Building Gaussian Mixture VAE using pytorch

I am trying to implement GMM-VAE model using torch. Basically I have a problem implementing for instance equation (1c) where the posterior distribution $p(x|z,w)$ is a Gaussian with the mean and ...
1
vote
1answer
76 views

What, exactly, do mlp(64,64) and mlp(64,128,1024) mean in PointNet, and how many input neurons does 1 (x,y,z) point have?

I couldn't find out how to interpret the multilayer perceptron notation given in PointNet. Specifically, I am looking to find out what the numbers inside the parentheses of ...
0
votes
0answers
6 views

The ratio between number of units in multi-input model

I have the model that accepts two inputs: Image from camera Speed of the car I can create some CNN layers to process the image input and some MLP layers to process other type of data (for example ...
0
votes
0answers
19 views

What's the effect of increasing hidden nodes?

Topic Demarcation I find many topics on "how to choose the number of hidden nodes". I'm not interested in the answer to that question. What I learned I learned, that if you increase the ...
1
vote
1answer
50 views

Why doesn't the LSTM model improve the time-series forecasting significantly with respect to the MLP model?

I have recently started learning time series forecasting. I have a dataset of the weekly payment history of 10k clients over 1 year, and I want to predict the future 5 payments for a test set of 1k ...
0
votes
0answers
24 views

Keras MLP performing better than Transformers

I'm working on a comparative study using some models in a sentiment analysis task: MLPs and LSTMs with and without the use of word embeddings (GloVe and Word2Vec) and two Transformer-based models (...
0
votes
0answers
23 views

GAN - relation of input size and hidden layer size

I'm adapting a GAN described here used for generating binary output. It's trained on binarized MNIST data, with a size of 28x28 so 784 values. I want to adapt it to train on and generate 1D vectors ...
0
votes
0answers
36 views

Why am I not getting a good accuracy but bad precision and recall for this binary classification problem (with an unbalanced dataset)?

I'm working on a simple MLP classification problem where I need to have the output layer as softmax. The dataset that I've been using needs to pass through a parse that I made to remove NaN and change ...
1
vote
1answer
60 views

About the choice of the activation functions in the Multilayer Perceptron, and on what does this depends?

I've read in this: F. Rosenblatt, Principles of neurodynamics. perceptrons and the theory of brain mechanisms that in the Multilayer Perceptron the activation functions in the second, third, ..., are ...
1
vote
0answers
47 views

What are the math theorems regarding the Multilayer Perceptron?

I've come across a theorem "Convergence theorem Simple Perceptron" for the first time, here-> https://zaguan.unizar.es/record/69205/files/TAZ-TFG-2018-148.pdf, page 27, (is in Spanish) ...
3
votes
1answer
138 views

Why is the backpropagation algorithm used to train the multilayer perceptron?

I've read in the book Neural Network Design, by Martin Hagan et al. (chapter 11), that, to train the feed-forward neural network (aka multilayer perceptron), one uses the backpropagation algorithm. ...
0
votes
0answers
118 views

What is the difference between the forward pass of the Multi-Layer Perceptron, Deep AutoEncoder and Deep Belief Network?

Multi-Layer Perceptron (MLP), Deep AutoEncoder (DAE), and Deep Belief Network (DBN) are trained differently. However, do they follow the same process during the inference phase, i.e., do they ...
1
vote
1answer
53 views

Can RNNs get inputs and produce outputs similar to the inputs and outputs of FFNNs?

RNN and LSTM models have many interesting architectures that can be modified in various ways. We can also compose their input and output data in quite interesting ways. However, in the examples that I ...
2
votes
1answer
84 views

What are examples of good free books that cover the back-propagation algorithm?

What are examples of good free books that cover the back-propagation used to train multilayer perceptrons? I've just started to learn about artificial neural networks, so I'm looking for books that ...
5
votes
1answer
374 views

Why do feedforward neural networks require the inputs to be of a fixed size, while RNNs can process variable-size inputs?

Why does a vanilla feedforward neural network only accept a fixed input size, while RNNs are capable of taking a series of inputs with no predetermined limit on the size? Can anyone elaborate on this ...
12
votes
0answers
186 views

What is the number of neurons required to approximate a polynomial of degree n?

I learned about the universal approximation theorem from this guide. It states that a network even with a single hidden layer can approximate any function within some bound, given a sufficient number ...
1
vote
0answers
33 views

How to draw a 3-dimensonal shape's neural network

I am reading an exam question about NN (that I cannot publish, for copyright reasons). The question says: 'Construct a rectangle in 2D space. Define the lines, and then define the weights and ...
0
votes
0answers
16 views

NN for multivariate function interpolation

I have a multivariate function. It takes 4, real valued inputs: a, b, c, d and returns 1 complex number, z. I wanted to use Neural Networks to predict the value z' for a generic input a', b', c', d'. ...
1
vote
0answers
16 views

Why does 0.8:0.2 divided dataset have a much greater AUROC than its 5-fold cross validated counterpart's mean AUROC?

I trained a dataset with a 5-fold cross validation to search for hyper-parameters by an AUROC metric. For splitting I used ...
2
votes
2answers
199 views

Why don't neural networks project the data into higher dimensions first, then reduce the size of each layer thereafter?

Background From my understanding (and following along with this blog post), (deep) neural networks apply transformations to the data such that the data's representation to the next layer (or ...
0
votes
1answer
316 views

Keras DQN Model with Multiple Inputs and Multiple Outputs [closed]

I am trying to create a DQN agent where I have 2 inputs: the agent's position and a matrix of 0s and 1s. The output is composed of the agent's new chosen position, a matrix of 0s and 1s (different ...
4
votes
1answer
65 views

Is there a common way to build a neural network that seeks to extract spatial and temporal information simultaneously?

Is there a common way to build a neural network that seeks to extract spatial and temporal information simultaneously? Is there an agreed up protocol on how to extract this information? What ...
2
votes
2answers
166 views

How is the error calculated with multiple output neurons in the neural network?

Machine Learning books generally explains that the error calculated for a given sample $i$ is: $e_i = y_i - \hat{y_i}$ Where $\hat{y}$ is the target output and $y$ is the actual output given by the ...
3
votes
1answer
365 views

Why does every neuron in hidden layers of a multi-layer perceptron typically have the same activation function? [duplicate]

Why does every neuron in a hidden layer of a multi-layer perceptron (MLP) typically have the same activation function as every other neuron in the same or other hidden layers (so I exclude the output ...
0
votes
1answer
823 views

Why can't MLPs perform non-linear regression and classification?

In this page it's told: In Single Perceptron / Multi-layer Perceptron(MLP), we only have linear separability because they are composed of input and output layers(some hidden layers in MLP) What ...
2
votes
2answers
92 views

What are standard datasets for fully connected neural networks?

I am looking for datasets that are used as a testing standard in the fully connected neural networks (FCNN). For example, in the image recognition and CNN, CIFAR datasets are used in most of the ...
1
vote
0answers
28 views

How can I adapt this script (to test the robustness of a perceptron) to test the robustness of a multi-layer perceptron?

The following script is from Trappenberg's Fundamentals of Computational Neuroscience and is used to test a perceptron's robustness against noise. However, how would one alter it to test the output ...
2
votes
2answers
295 views

Why is it called back-propagation?

While looking at the mathematics of the back-propagation algorithm for a multi-layer perceptron, I noticed that in order to find the partial derivative of the cost function with respect to a weight (...
5
votes
2answers
300 views

Can neurons in MLP and filters in CNN be compared?

I know they are not the same in working, but an input layer sends the input to $n$ neurons with a set of weights, based on these weights and the activation layer, it produces an output that can be fed ...
4
votes
1answer
64 views

Why MLP cannot approximate a closed shape function?

[TL;DR] I generated two classes Red and Blue on a 2D space. Red are points on Unit Circle and Blue are points on a Circle Ring with radius limits (3,4). I tried to train a Multi Layer Perceptron ...
2
votes
0answers
45 views

Recent algorithms for correcting mislabeled data using multilayer perceptrons

I am doing literature research on algorithms for correcting mislabeled data using multilayer perceptrons. Found an "old" paper An algorithm for correcting mislabeled data (2001) by Xinchuan Zeng et al....
4
votes
1answer
84 views

Are the labels updated during training in the algorithm presented in "An algorithm for correcting mislabeled data"?

I am trying to understand an algorithm for correcting mislabeled data in the paper An algorithm for correcting mislabeled data (2001) by Xinchuan Zeng et al. The authors are suggesting to update the ...
1
vote
0answers
36 views

Model unfit for some part of spiral data despite low error

I'm current testing a model for spiral data. After 500 epoches, loss is 0.04 but the result is still unmatch with some part of the training data. (bottom left) The model has 2 hidden tanh x 16 units ...
2
votes
1answer
130 views

Is there a mathematical theory behind why MLP can classify handwritten digits?

I'm trying to really understand how multi-layer perceptrons work. I want to prove mathematically that MLP's can classify handwritten digits. The only thing I really have is that each perceptron can ...
1
vote
0answers
25 views

Keras MLP returns always loss 0.0 [closed]

I'm implementing a multilayer perceptron with Keras to predict the correct words order in a sentence. I'm using train_on_batch()because I convert each sentence in a ...
2
votes
0answers
192 views

How I can predict the next number in a sequence with a neural network?

I've been dabbling with machine learning and neural networks (namely, resnet50) for a few months now, mostly doing image recognition. I am currently trying to make a program that, given a string of ...
1
vote
1answer
89 views

Why does my model overfit on pseudo-random numbers training data?

I am trying to predict pseudo-random numbers using the past numbers with a multiplayer perceptron. The error while training is very low. However, as soon as I test it with a test set, the model ...
4
votes
1answer
84 views

Is it expected that adding an additional hidden layer to my 3-layer ANN reduces accuracy significantly?

I've been using several resources to implement my own artificial neural network package in C++. Among some of the resources I've been using are https://www.anotsorandomwalk.com/backpropagation-...
1
vote
0answers
31 views

Does a varying ANN model accuracy mean underfitting or overfitting?

Background: This is for a simulated robot with four legs, walking on a flat terrain. The ANN (an MLP) is given inputs as the robot's body angle, positions and angle of each leg with respect to the ...
1
vote
0answers
29 views

Applying Machine Learning to 2D Laser Scanner Data

We are using 2D Laser Scanner to scan various objects of different geometric shapes for e.g. cylinder, spiked, cylinder with notch, cylinder with curved edges e.t.c. The dataset contains points in the ...
8
votes
2answers
1k views

Why use a recurrent neural network over a feedforward neural network for sequence prediction?

If recurrent neural networks (RNNs) are used to capture prior information, couldn't the same thing be achieved by a feedforward neural network (FFNN) or multi-layer perceptron (MLP) where the inputs ...
1
vote
1answer
138 views

How can I access the weights at each training iteration of an MLP with scikit-learn?

I'm building an MLP with scikit-learn. Is there a way I can access the weights and biases of the output layer per iteration? There is an option mlp.coefs_, but it ...
1
vote
1answer
80 views

How can we print weights per iteration in a simple feed forward MLP for an specific class?

im working on a project in which I have to make a multi-layer perceptron with two hidden layers with 3 nodes in each. The target value in my data contains 8 unique values/classes. One of the tasks ...
1
vote
0answers
46 views

Is there data available about successful neural network architectures?

I am curious to if there is data available for MLP architectures in use today, their initial architecture, the steps that were taken to improve the architecture to an acceptable state and what the ...
2
votes
3answers
312 views

One vs multiple output neurons

Consider an MLP that outputs an integer 'rating' of 0 to 4. Would it be correct to say this could be modeled in either of the following ways: map each rating in the dataset to a 'normalized set' ...
4
votes
1answer
102 views

Backpropagation equation for a variant on the usual Linear Neuron architecture

Recently I encountered a variant on the normal linear neural layer architecture: Instead of $Z = XW + B$, we now have $Z = (X-A)W + B$. So we have a 'pre-bias' $A$ that affects the activation of the ...