Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [multilayer-perceptrons]

For question about Multi Layer Perceptron model/architecture, its training and other related details and parameters associated with the model.

Filter by
Sorted by
Tagged with
1 vote
0 answers
17 views

How a MLP learns from images and boundaries of its capabilities

(Sorry, I'm not very experienced with ML and I apologize if these questions are vague, naive, or not written well. I also moved this from CV to here.) I think I understand that a MLP learns (via ...
riley's user avatar
  • 11
0 votes
0 answers
13 views

Proof that Temporal-Difference TD(1) is Equivalent to Widrow-Hoff

I'm reading Sutton's "Learning to Predict by the Methods of Temporal Differences" and I'm getting hung up on a derivation (p. 14). We are considering (observation-sequence, outcome) pairs. $[...
Matheo Xenakis's user avatar
0 votes
0 answers
24 views

Neural Network with Incorrect Calculation Better than Correct One

I have designed my own neural network and discovered an error. During backpropagation, instead of inserting the Z-values into the derivative of the activation function, I inserted the A-values. The ...
Apro9991's user avatar
0 votes
1 answer
44 views

Xavier vs He initialization with tanh

I'm a student and in the lecture, I learned that He initialization is better than Xavier if you use ReLU activation function. In addition, I also learned that Xavier initialization is better than He ...
COTHE's user avatar
  • 5
0 votes
1 answer
51 views

Total loss in backpropagation

I'd say I have some understanding of backpropagation, however I am not really sure of the total loss being calculated. Let us take the example below: After 1 forward pass when I have to update the ...
xkcd101's user avatar
0 votes
1 answer
42 views

How to optimise a FNN/MLP network with MSE (positive only loss) in C

I can create a FNN/MLP network in C but only g-p loss works, where g = ground truth and p = predicted. What I don't understand is how MSE a positive only loss value can train a back propagation ...
user avatar
2 votes
1 answer
67 views

Is it a requirement/recommendation to normalize my inputs into [0,1] range?

All features of my input dataset, which is going to be used for training a simple multi-layered neural network, are in range $[-1,+1]$ and the output of $NN$ is a single number again in range $[-1,+1]$...
Bikay's user avatar
  • 23
0 votes
0 answers
8 views

Cannot find appropriate model to classify hidden states

My input data is vector representing encoded image - 22 features, and I try to classify by 3 classes 0, 1, 2 (neutral, good, bad) Original: ...
Max Usanin's user avatar
0 votes
1 answer
37 views

Validation loss is always lower than training loss whatever i try

I've been training several types of MLPs with different optimisers and tuned them with keras's hyperband tuner. All of them follow this cone architecture: All the networks were trained on the same ...
Roger Smith's user avatar
0 votes
2 answers
64 views

How are hidden layers counted / semantically defined?

I'm working my way through how LLMs work and I understand how things work but it's not clear to me exactly what is semantically defined as a "layer". Using the following FFN as an example: ...
Grant Curell's user avatar
0 votes
1 answer
60 views

Is it possible training accuracy never changed while training?

Question summary What informations can get from this epoch_accuracy graph? Is it possible training accuracy never changed like after 10 epoch in graph while training? Body I do some experiments with ...
Yang's user avatar
  • 57
0 votes
0 answers
24 views

Permute inputs of neural network and expect different outputs?

My question is about if it makes sense to define a function (MLP) that takes two feature vectors f1 and f2. However, I want MLP(f1, f2) != MLP(f2, f1). I believe ...
mra-h's user avatar
  • 1
0 votes
0 answers
56 views

Why does my loss function fluctuate so much?

I have a loss function that I'm trying to maximise using a neural network. While it does appear to increase and plateau over the training, it does so in a very "noisy" manner, spiking up and ...
VJ123's user avatar
  • 73
0 votes
1 answer
27 views

Can a concept/feature be represented using more than one layer of a Neural Network?

I was reading Goodfellow. At the start of the text it was mentioned that there are two ways to represent depth of a deep neural network. One is using the depth of the computation graph and the other ...
rsonx's user avatar
  • 101
2 votes
1 answer
333 views

Calculating mutual information between layer outputs and targets in a neural network

I've seen in several papers that it is possible to calculate the mutual information between a layer's outputs and the desired outputs. For example: Source: https://www.ncbi.nlm.nih.gov/pmc/articles/...
VJ123's user avatar
  • 73
1 vote
1 answer
97 views

Why does averaging attention-weighted positions reduce the effective resolution in transformers?

I was reading this blog post from Harvard and it says in its background paragraph about transformers that the number of operations required to relate signals from two arbitrary input or output ...
Daviiid's user avatar
  • 575
0 votes
0 answers
95 views

How to generate original training videos based on existing videoset?

I am a software engineer who is quickly ramping up on AI tech, but am nevertheless very new to the sector. A collegue has an extensive collection of training videos, the vertical is wheelchair seating ...
lukabloomrox's user avatar
3 votes
1 answer
499 views

Are transformer models better than comparable-complexity MLP-based models?

I've watched the outstanding Andrej Karpathy's From Zero to Hero course. In the last lecture, he introduces Transformer decoder architecture, which is able to produce Shakespear-like text. However, ...
DeLorean88's user avatar
1 vote
0 answers
178 views

How can an MLP be implemented with convolutional layers?

I am studying the architecture of the network pointnet, specifically the MLPs stages of the pipeline highlighted in red in the following image (taken from the author page here): It is strange to find ...
Jacob Morales Gonzalez's user avatar
3 votes
0 answers
55 views

Why does a neural network struggle to solve this simple problem?

Consider the following problem: Given a vector x of size dim with values between 0 and 1 (exclusive), determine if ...
Daniel's user avatar
  • 191
0 votes
0 answers
46 views

Are autoencoders computationally cheaper than MLPs with the same number of neurons?

Are autoencoders computationally cheaper than other neural networks such as MLP with the same number of neurons? I have read in some papers that autoencoders train the network faster, and I could ...
Jesus M.'s user avatar
2 votes
2 answers
153 views

Why is a simple regression problem so hard for an MLP to learn?

Consider a very simple problem, which is to find the maximum value out of a list of 5 numbers between 0 and 1. This is obviously trivial, but serves as a good example for a real-world problem I'm ...
Daniel's user avatar
  • 191
3 votes
1 answer
135 views

Why is automatic differentiation still used, if today's computers can calculate symbolic derivatives quite fast?

Today's computers can calculate symbolic derivatives quite fast, why is automatic differentiation still used? For example, Mathematica can handle algebraic operations with arrays. Doesn't automatic ...
asd's user avatar
  • 33
2 votes
1 answer
128 views

Multi-objective training involving maximization of one loss function and minimization of another

I need my model to predict $s$ from my data $x$. Additionally, I need the model to not use signals in $x$ that are predictive of a separate target $a$. My approach is to transform $x$ into a ...
ChargeShivers's user avatar
0 votes
2 answers
281 views

Concise and mathematically-oriented book on AI and neural networks suitable as a gift [closed]

I would like to buy a book about AI and neural networks written on accessible level for a 17 years old mathematically very gifted student interested in these topics. The book should contain some ...
yarchik's user avatar
  • 67
1 vote
2 answers
62 views

What are all the possible usages of 'multilayer perceptron'?

The term 'multilayer perceptron' has been used in literature in various ways in the literature. I am presenting some of them below As a feed-forward neural network [1]. As a fully connected feed-...
hanugm's user avatar
  • 3,890
2 votes
1 answer
79 views

How to decode P bits that represent a random weight generator?

So I've been tasked by my neural network professor at university to replicate the following research: Intelligent Breast Cancer Diagnosis Using Hybrid GA-ANN. Each chromosome represents a possible net,...
JOSEPH CAROÈ's user avatar
0 votes
1 answer
32 views

Do Quo et al (2013) perform backpropagation between layers?

Le et al. 2013's non-weight sharing CNN has inspired me to ask two questions on this site previously. When training the three-layer autoencoder, do they compute dL/dW (where L is equation 1) ...
Josiah Yoder's user avatar
0 votes
1 answer
419 views

How to make a proper approximation of Sine function with Neural Networks?

TL;DR; How to build a neural network that properly approximates the sine function with different ranges? Context and Question: From this question I decided to use the Sergey's answer, however I used a ...
Hans's user avatar
  • 111
1 vote
1 answer
87 views

Does a second-order fully-connected layer have any uses?

I was thinking about implementing second-order regression via a fully-connected layer, and I came up with this: $X$ is the input data, shaped $(features, batch\_number)$. $w0$ is the bias, shaped $(...
HappyFace's user avatar
  • 113
1 vote
1 answer
95 views

Why is training all layers at a time effective for a multi-layer autoencoder?

This training of all layers of a CNN simultaneously is standard practice today. It is found in every CNN (AlexNet (2012), VGG, Inception, GANs, etc) and even pre-CNN networks such as Le et al. 2012. ...
Josiah Yoder's user avatar
1 vote
0 answers
76 views

Is the capability of RNN more than the capability of MLP?

Consider the following excerpt paragraph taken from the section titled "Recurrent Neural Networks" of the chapter 10: Sequence Modeling: Recurrent and Recursive Nets of the textbook named ...
hanugm's user avatar
  • 3,890
1 vote
0 answers
74 views

Does Godel's incompleteness theorems restricts the scope of connectionist-AI?

It is well-known that Godel's incompleteness theorems restricted the reachability of symbolic-AI, which is dependent on mathematical logic. But, I am wondering whether it has any impact on the ...
hanugm's user avatar
  • 3,890
1 vote
1 answer
203 views

Are the capabilities of connectionist AI and symbolic AI the same?

The universal approximation theorem says that MLP with a single hidden layer and enough number of neurons can able to approximate any bounded continuous function. You can validate it from the ...
hanugm's user avatar
  • 3,890
1 vote
1 answer
83 views

Rank of gradient-of-loss with respect to layer weights in an MLP

The paper: https://arxiv.org/abs/2110.11309, makes the following claim at the end of page 3: The gradient of loss $L$ with respect to weights $W_l$ of an MLP is a rank-1 matrix for each of B batch ...
Andrew's user avatar
  • 121
1 vote
0 answers
74 views

Is the VC dimension of a MLP regressor a valid upper bound on how many points it can exactly fit?

I want to calculate an upper bound on how many training points an MLP regressor can fit with ~0 error. I don't care about the test error, I want to overfit as much as possible the (few) training ...
Daniele 's user avatar
1 vote
1 answer
294 views

When should we use CNN instead of MLP?

Is CNN only applicable to time-series data or image data? When should we use CNN instead of MLP?
user366312's user avatar
0 votes
1 answer
2k views

What, exactly, do mlp(64,64) and mlp(64,128,1024) mean in PointNet, and how many input neurons does 1 (x,y,z) point have?

I couldn't find out how to interpret the multilayer perceptron notation given in PointNet. Specifically, I am looking to find out what the numbers inside the parentheses of ...
Justin's user avatar
  • 41
1 vote
1 answer
566 views

Why doesn't the LSTM model improve the time-series forecasting significantly with respect to the MLP model?

I have recently started learning time series forecasting. I have a dataset of the weekly payment history of 10k clients over 1 year, and I want to predict the future 5 payments for a test set of 1k ...
Varazda's user avatar
  • 111
0 votes
1 answer
112 views

About the choice of the activation functions in the Multilayer Perceptron, and on what does this depends?

I've read in this: F. Rosenblatt, Principles of neurodynamics. perceptrons and the theory of brain mechanisms that in the Multilayer Perceptron the activation functions in the second, third, ..., are ...
Verónica Rmz.'s user avatar
0 votes
0 answers
71 views

What are the math theorems regarding the Multilayer Perceptron?

I've come across a theorem "Convergence theorem Simple Perceptron" for the first time, here-> https://zaguan.unizar.es/record/69205/files/TAZ-TFG-2018-148.pdf, page 27, (is in Spanish) ...
Verónica Rmz.'s user avatar
2 votes
1 answer
424 views

Why is the backpropagation algorithm used to train the multilayer perceptron?

I've read in the book Neural Network Design, by Martin Hagan et al. (chapter 11), that, to train the feed-forward neural network (aka multilayer perceptron), one uses the backpropagation algorithm. ...
Verónica Rmz.'s user avatar
0 votes
0 answers
169 views

What is the difference between the forward pass of the Multi-Layer Perceptron, Deep AutoEncoder and Deep Belief Network?

Multi-Layer Perceptron (MLP), Deep AutoEncoder (DAE), and Deep Belief Network (DBN) are trained differently. However, do they follow the same process during the inference phase, i.e., do they ...
witdev's user avatar
  • 73
1 vote
1 answer
84 views

Can RNNs get inputs and produce outputs similar to the inputs and outputs of FFNNs?

RNN and LSTM models have many architectures that can be modified. We can also compose their input and output data. However, in the examples that I found on the web, the inputs and outputs of RNNs/...
Green's user avatar
  • 45
1 vote
1 answer
549 views

What are examples of good free books that cover the back-propagation algorithm?

What are examples of good free books that cover the back-propagation used to train multilayer perceptrons? I've just started to learn about artificial neural networks, so I'm looking for books that ...
Verónica Rmz.'s user avatar
6 votes
1 answer
1k views

Why do feedforward neural networks require the inputs to be of a fixed size, while RNNs can process variable-size inputs?

Why does a vanilla feedforward neural network only accept a fixed input size, while RNNs are capable of taking a series of inputs with no predetermined limit on the size? Can anyone elaborate on this ...
Daniel's user avatar
  • 63
19 votes
1 answer
1k views

What is the number of neurons required to approximate a polynomial of degree n?

I learned about the universal approximation theorem from this guide. It states that a network even with a single hidden layer can approximate any function within some bound, given a sufficient number ...
mark mark's user avatar
  • 763
0 votes
0 answers
83 views

How to draw a 3-dimensonal shape's neural network

I am reading an exam question about NN (that I cannot publish, for copyright reasons). The question says: 'Construct a rectangle in 2D space. Define the lines, and then define the weights and ...
Slowat_Kela's user avatar
2 votes
2 answers
988 views

Why don't neural networks project the data into higher dimensions first, then reduce the size of each layer thereafter?

Background From my understanding (and following along with this blog post), (deep) neural networks apply transformations to the data such that the data's representation to the next layer (or ...
Kevin's user avatar
  • 133
0 votes
1 answer
750 views

Keras DQN Model with Multiple Inputs and Multiple Outputs [closed]

I am trying to create a DQN agent where I have 2 inputs: the agent's position and a matrix of 0s and 1s. The output is composed of the agent's new chosen position, a matrix of 0s and 1s (different ...
Ness's user avatar
  • 206