Questions tagged [multilayer-perceptron]

For question about Multi Layer Perceptron model/architecture, its training and other related details and parameters associated with the model.

Filter by
Sorted by
Tagged with
9
votes
0answers
115 views

What is the number of neurons required to approximate a polynomial of degree n?

I learned about the universal approximation theorem from this guide. It states that a network even with a single hidden layer can approximate any function within some bound, given a sufficient number ...
8
votes
2answers
1k views

Why use a recurrent neural network over a feedforward neural network for sequence prediction?

If recurrent neural networks (RNNs) are used to capture prior information, couldn't the same thing be achieved by a feedforward neural network (FFNN) or multi-layer perceptron (MLP) where the inputs ...
8
votes
2answers
2k views

Did Minsky and Papert know that multi-layer perceptrons could solve XOR?

In their famous book entitled Perceptrons: An Introduction to Computational Geometry, Minsky and Papert show that a perceptron can't solve the XOR problem. This contributed to the first AI winter, ...
6
votes
4answers
657 views

What are some datasets to train an MLP on simple tasks? [closed]

I have implemented an MLP. Now, I want to train it to solve simple tasks. Are there any data sets to train the MLP on simple tasks, that is, tasks with a small number of inputs and outputs? I ...
5
votes
2answers
211 views

Is a multilayer perceptron a recursive function?

I read somewhere that a multilayer perceptron is a recursive function in its forward propagation phase. I am not sure, what is the recursive part? For me, I would see an MLP as a chained function. So, ...
5
votes
1answer
141 views

How does a single hidden layer affect output? [duplicate]

I'm learning about multilayer perceptrons, and I have a quick theory question in regards to hidden layer neurons. I know we can use two hidden layers to solve a non-linearly separable problem by ...
4
votes
1answer
218 views

Why do feedforward neural networks require the inputs to be of a fixed size, while RNNs can process variable-size inputs?

Why does a vanilla feedforward neural network only accept a fixed input size, while RNNs are capable of taking a series of inputs with no predetermined limit on the size? Can anyone elaborate on this ...
4
votes
2answers
145 views

Can neurons in MLP and filters in CNN be compared?

I know they are not the same in working, but an input layer sends the input to $n$ neurons with a set of weights, based on these weights and the activation layer, it produces an output that can be fed ...
4
votes
1answer
58 views

Why MLP cannot approximate a closed shape function?

[TL;DR] I generated two classes Red and Blue on a 2D space. Red are points on Unit Circle and Blue are points on a Circle Ring with radius limits (3,4). I tried to train a Multi Layer Perceptron ...
4
votes
1answer
151 views

Is it a valid assumption that a purely MLP based tic-tac-toe player will learn lookahead strategies?

I'm doing a little tic-tac-toe project to learn neural networks and machine learning (beginner level). I've written a MLP based program that plays with other search based programs and trains with the ...
4
votes
1answer
70 views

Are the labels updated during training in the algorithm presented in “An algorithm for correcting mislabeled data”?

I am trying to understand an algorithm for correcting mislabeled data in the paper An algorithm for correcting mislabeled data (2001) by Xinchuan Zeng et al. The authors are suggesting to update the ...
4
votes
1answer
78 views

Is it expected that adding an additional hidden layer to my 3-layer ANN reduces accuracy significantly?

I've been using several resources to implement my own artificial neural network package in C++. Among some of the resources I've been using are https://www.anotsorandomwalk.com/backpropagation-...
4
votes
1answer
96 views

Backpropagation equation for a variant on the usual Linear Neuron architecture

Recently I encountered a variant on the normal linear neural layer architecture: Instead of $Z = XW + B$, we now have $Z = (X-A)W + B$. So we have a 'pre-bias' $A$ that affects the activation of the ...
4
votes
1answer
114 views

How to deal with padded inputs in a fully connected feed forward network?

I have a fully connected network that takes in a variable length input padded with 0. However the network doesn't seem to be learning and I am guessing that the high number of zeros in the input ...
3
votes
2answers
461 views

How can a neural network learn to play sudoku?

I'm just beginning to understand neural networks and I've performed a couple of successful tests with numerical series where the NN was trained to find the odd one or a missing value. It all works ...
3
votes
1answer
60 views

Is there a common way to build a neural network that seeks to extract spatial and temporal information simultaneously?

Is there a common way to build a neural network that seeks to extract spatial and temporal information simultaneously? Is there an agreed up protocol on how to extract this information? What ...
3
votes
1answer
222 views

Why does every neuron in hidden layers of a multi-layer perceptron typically have the same activation function? [duplicate]

Why does every neuron in a hidden layer of a multi-layer perceptron (MLP) typically have the same activation function as every other neuron in the same or other hidden layers (so I exclude the output ...
3
votes
3answers
2k views

Why MLP momentum term must be in the range 0-1?

Why momentum factor greater than 1 is a bad idea? What are the mathematical conclusions?
3
votes
2answers
104 views

Which online machine learning technique to use for multi-class classification problem with multiple inputs?

I have the following problem. We have $4$ separate discrete inputs, which can take any integer value between $-63$ and $63$. The output is also supposed to be a discrete value between $-63$ and $63$. ...
3
votes
2answers
328 views

Unable to overfit using MLP

I'm building a 5-class classifier with a private dataset. Each data sample has 67 features and there are about 40000 samples. Samples of a particular class were duplicated to overcome class imbalance ...
2
votes
2answers
211 views

Why is it called back-propagation?

While looking at the mathematics of the back-propagation algorithm for a multi-layer perceptron, I noticed that in order to find the partial derivative of the cost function with respect to a weight (...
2
votes
1answer
102 views

Why is the Backpropagation algorithm used to train the Multilayer Perceptron?

I've read on the book NND by Martin Hagan et al (chapter 11), that to train the feed-forward neural network: Multilayer Perceptron one uses the Backpropagation algorithm. Why this algorithm? Could ...
2
votes
2answers
65 views

What are standard datasets for fully connected neural networks?

I am looking for datasets that are used as a testing standard in the fully connected neural networks (FCNN). For example, in the image recognition and CNN, CIFAR datasets are used in most of the ...
2
votes
1answer
231 views

Using a MLP to predict a 12x12 matrix

So, i need to use an MLP to predict a 12x12 matrix composed of floating points. The matrices are as this one that follows: Most matrices have this "pattern". As input, i have 7 floating points, ...
2
votes
3answers
256 views

One vs multiple output neurons

Consider an MLP that outputs an integer 'rating' of 0 to 4. Would it be correct to say this could be modeled in either of the following ways: map each rating in the dataset to a 'normalized set' ...
2
votes
1answer
60 views

Why are activation functions independent layers in CNNs rather than part of convolutional layers?

I have been reading up on CNNs. One of the different confusing things has been that people always talk of normalization layers. A common normalization layer is a ReLU layer. But I never encountered an ...
2
votes
0answers
44 views

Recent algorithms for correcting mislabeled data using multilayer perceptrons

I am doing literature research on algorithms for correcting mislabeled data using multilayer perceptrons. Found an "old" paper An algorithm for correcting mislabeled data (2001) by Xinchuan Zeng et al....
2
votes
0answers
166 views

How I can predict the next number in a sequence with a neural network?

I've been dabbling with machine learning and neural networks (namely, resnet50) for a few months now, mostly doing image recognition. I am currently trying to make a program that, given a string of ...
2
votes
1answer
126 views

Is there a mathematical theory behind why MLP can classify handwritten digits?

I'm trying to really understand how multi-layer perceptrons work. I want to prove mathematically that MLP's can classify handwritten digits. The only thing I really have is that each perceptron can ...
1
vote
1answer
232 views

Comments on my proposed “Jitter” neuron

I have an application of neural networks (standard MLP architecture) where I want to forecast a tanh output (ranging from -1 to +1) with about 1500 input features in ~700 samples. Each sample ...
1
vote
2answers
78 views

How is the error calculated with multiple output neurons in the neural network?

Machine Learning books generally explains that the error calculated for a given sample $i$ is: $e_i = y_i - \hat{y_i}$ Where $\hat{y}$ is the target output and $y$ is the actual output given by the ...
1
vote
1answer
73 views

How can I access the weights at each training iteration of an MLP with scikit-learn?

I'm building an MLP with scikit-learn. Is there a way I can access the weights and biases of the output layer per iteration? There is an option mlp.coefs_, but it ...
1
vote
1answer
41 views

Can RNNs get inputs and produce outputs similar to the inputs and outputs of FFNNs?

RNN and LSTM models have many interesting architectures that can be modified in various ways. We can also compose their input and output data in quite interesting ways. However, in the examples that I ...
1
vote
1answer
60 views

What are examples of good free books that cover the back-propagation algorithm?

What are examples of good free books that cover the back-propagation used to train multilayer perceptrons? I've just started to learn about artificial neural networks, so I'm looking for books that ...
1
vote
1answer
74 views

Why does my model overfit on pseudo-random numbers training data?

I am trying to predict pseudo-random numbers using the past numbers with a multiplayer perceptron. The error while training is very low. However, as soon as I test it with a test set, the model ...
1
vote
1answer
73 views

How can we print weights per iteration in a simple feed forward MLP for an specific class?

im working on a project in which I have to make a multi-layer perceptron with two hidden layers with 3 nodes in each. The target value in my data contains 8 unique values/classes. One of the tasks ...
1
vote
2answers
94 views

Why don't neural networks project the data into higher dimensions first, then reduce the size of each layer thereafter?

Background From my understanding (and following along with this blog post), (deep) neural networks apply transformations to the data such that the data's representation to the next layer (or ...
1
vote
0answers
27 views

How can I adapt this script (to test the robustness of a perceptron) to test the robustness of a multi-layer perceptron?

The following script is from Trappenberg's Fundamentals of Computational Neuroscience and is used to test a perceptron's robustness against noise. However, how would one alter it to test the output ...
1
vote
0answers
26 views

Model unfit for some part of spiral data despite low error

I'm current testing a model for spiral data. After 500 epoches, loss is 0.04 but the result is still unmatch with some part of the training data. (bottom left) The model has 2 hidden tanh x 16 units ...
1
vote
0answers
24 views

Keras MLP returns always loss 0.0 [closed]

I'm implementing a multilayer perceptron with Keras to predict the correct words order in a sentence. I'm using train_on_batch()because I convert each sentence in a ...
1
vote
0answers
29 views

Does a varying ANN model accuracy mean underfitting or overfitting?

Background: This is for a simulated robot with four legs, walking on a flat terrain. The ANN (an MLP) is given inputs as the robot's body angle, positions and angle of each leg with respect to the ...
1
vote
0answers
26 views

Applying Machine Learning to 2D Laser Scanner Data

We are using 2D Laser Scanner to scan various objects of different geometric shapes for e.g. cylinder, spiked, cylinder with notch, cylinder with curved edges e.t.c. The dataset contains points in the ...
1
vote
0answers
46 views

Is there data available about successful neural network architectures?

I am curious to if there is data available for MLP architectures in use today, their initial architecture, the steps that were taken to improve the architecture to an acceptable state and what the ...
1
vote
0answers
70 views

How can an ANN efficiently predict multiple numbers with fixed sum (in other words, proportions)?

I need a neural network (or any other solution) to predict 3 values which sum equals a fixed number (100). This will help me calculate proportions. Which is the most efficient way to do this? The ...
1
vote
0answers
213 views

Sigmoid output layer and Cross-Entropy cost function

I use Sigmoid activation function for neurons at output layer of my Multi-Layer Perceptron also, I use cross-entropy cost function. As I know when activation functions like Tanh is used in output ...
0
votes
1answer
48 views

About the choice of the activation functions in the Multilayer Perceptron, and on what does this depends?

I've read in this: F. Rosenblatt, Principles of neurodynamics. perceptrons and the theory of brain mechanisms that in the Multilayer Perceptron the activation functions in the second, third, ..., are ...
0
votes
1answer
166 views

Keras DQN Model with Multiple Inputs and Multiple Outputs [closed]

I am trying to create a DQN agent where I have 2 inputs: the agent's position and a matrix of 0s and 1s. The output is composed of the agent's new chosen position, a matrix of 0s and 1s (different ...
0
votes
1answer
438 views

Why can't MLPs perform non-linear regression and classification?

In this page it's told: In Single Perceptron / Multi-layer Perceptron(MLP), we only have linear separability because they are composed of input and output layers(some hidden layers in MLP) What ...
0
votes
0answers
5 views

Accuracy, F1 score and Recall outputting the same results with softmax

I'm working on a simple MLP classification problem where I need to have the output layer as Softmax. The dataset that I've been using needs to pass through a parse that I made to remove NaN and change ...
0
votes
0answers
40 views

What are the math theorems regarding the Multilayer Perceptron?

I've come across a theorem "Convergence theorem Simple Perceptron" for the first time, here-> https://zaguan.unizar.es/record/69205/files/TAZ-TFG-2018-148.pdf, page 27, (is in Spanish) ...