Questions tagged [multilayer-perceptrons]

For question about Multi Layer Perceptron model/architecture, its training and other related details and parameters associated with the model.

28 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3 votes
0 answers
50 views

Why does a neural network struggle to solve this simple problem?

Consider the following problem: Given a vector x of size dim with values between 0 and 1 (exclusive), determine if ...
Daniel's user avatar
  • 191
3 votes
2 answers
236 views

Which online machine learning technique to use for multi-class classification problem with multiple inputs?

I have the following problem. We have $4$ separate discrete inputs, which can take any integer value between $-63$ and $63$. The output is also supposed to be a discrete value between $-63$ and $63$. ...
John Mayer's user avatar
2 votes
1 answer
117 views

Multi-objective training involving maximization of one loss function and minimization of another

I need my model to predict $s$ from my data $x$. Additionally, I need the model to not use signals in $x$ that are predictive of a separate target $a$. My approach is to transform $x$ into a ...
ChargeShivers's user avatar
2 votes
1 answer
69 views

How to decode P bits that represent a random weight generator?

So I've been tasked by my neural network professor at university to replicate the following research: Intelligent Breast Cancer Diagnosis Using Hybrid GA-ANN. Each chromosome represents a possible net,...
JOSEPH CAROÈ's user avatar
2 votes
0 answers
749 views

How I can predict the next number in a sequence with a neural network?

I've been dabbling with machine learning and neural networks (namely, resnet50) for a few months now, mostly doing image recognition. I am currently trying to make a program that, given a string of ...
c0mraide's user avatar
2 votes
1 answer
69 views

Applying Machine Learning to 2D Laser Scanner Data

We are using 2D Laser Scanner to scan various objects of different geometric shapes for e.g. cylinder, spiked, cylinder with notch, cylinder with curved edges e.t.c. The dataset contains points in the ...
PMu's user avatar
  • 21
2 votes
2 answers
568 views

Unable to overfit using MLP

I'm building a 5-class classifier with a private dataset. Each data sample has 67 features and there are about 40000 samples. Samples of a particular class were duplicated to overcome class imbalance ...
Ashwin Kannan's user avatar
2 votes
1 answer
147 views

Is there a mathematical theory behind why MLP can classify handwritten digits?

I'm trying to really understand how multi-layer perceptrons work. I want to prove mathematically that MLP's can classify handwritten digits. The only thing I really have is that each perceptron can ...
user8714896's user avatar
1 vote
0 answers
36 views

Why does averaging attention-weighted positions reduce the effective resolution in transformers?

I was reading this blog post from Harvard and it says in its background paragraph about transformers that the number of operations required to relate signals from two arbitrary input or output ...
Daviiid's user avatar
  • 573
1 vote
0 answers
107 views

How can an MLP be implemented with convolutional layers?

I am studying the architecture of the network pointnet, specifically the MLPs stages of the pipeline highlighted in red in the following image (taken from the author page here): It is strange to find ...
Jacob Morales Gonzalez's user avatar
1 vote
0 answers
69 views

Is the capability of RNN more than the capability of MLP?

Consider the following excerpt paragraph taken from the section titled "Recurrent Neural Networks" of the chapter 10: Sequence Modeling: Recurrent and Recursive Nets of the textbook named ...
hanugm's user avatar
  • 3,612
1 vote
0 answers
67 views

Does Godel's incompleteness theorems restricts the scope of connectionist-AI?

It is well-known that Godel's incompleteness theorems restricted the reachability of symbolic-AI, which is dependent on mathematical logic. But, I am wondering whether it has any impact on the ...
hanugm's user avatar
  • 3,612
1 vote
1 answer
69 views

Rank of gradient-of-loss with respect to layer weights in an MLP

The paper: https://arxiv.org/abs/2110.11309, makes the following claim at the end of page 3: The gradient of loss $L$ with respect to weights $W_l$ of an MLP is a rank-1 matrix for each of B batch ...
Andrew's user avatar
  • 63
1 vote
0 answers
66 views

Is the VC dimension of a MLP regressor a valid upper bound on how many points it can exactly fit?

I want to calculate an upper bound on how many training points an MLP regressor can fit with ~0 error. I don't care about the test error, I want to overfit as much as possible the (few) training ...
Daniele 's user avatar
1 vote
0 answers
61 views

Model unfit for some part of spiral data despite low error

I'm current testing a model for spiral data. After 500 epoches, loss is 0.04 but the result is still unmatch with some part of the training data. (bottom left) (source: upsieutoc.com) The model has 2 ...
uahnbu's user avatar
  • 11
1 vote
0 answers
55 views

Does a varying ANN model accuracy mean underfitting or overfitting?

Background: This is for a simulated robot with four legs, walking on a flat terrain. The ANN (an MLP) is given inputs as the robot's body angle, positions and angle of each leg with respect to the ...
John David's user avatar
1 vote
0 answers
49 views

Is there data available about successful neural network architectures?

I am curious to if there is data available for MLP architectures in use today, their initial architecture, the steps that were taken to improve the architecture to an acceptable state and what the ...
SandMan's user avatar
  • 253
1 vote
0 answers
173 views

How can an ANN efficiently predict multiple numbers with fixed sum (in other words, proportions)?

I need a neural network (or any other solution) to predict 3 values which sum equals a fixed number (100). This will help me calculate proportions. Which is the most efficient way to do this? The ...
luvzfootball's user avatar
1 vote
0 answers
243 views

Sigmoid output layer and Cross-Entropy cost function

I use Sigmoid activation function for neurons at output layer of my Multi-Layer Perceptron also, I use cross-entropy cost function. As I know when activation functions like Tanh is used in output ...
CVDE's user avatar
  • 185
0 votes
2 answers
48 views

How are hidden layers counted / semantically defined?

I'm working my way through how LLMs work and I understand how things work but it's not clear to me exactly what is semantically defined as a "layer". Using the following FFN as an example: ...
Grant Curell's user avatar
0 votes
0 answers
22 views

Permute inputs of neural network and expect different outputs?

My question is about if it makes sense to define a function (MLP) that takes two feature vectors f1 and f2. However, I want MLP(f1, f2) != MLP(f2, f1). I believe ...
mra-h's user avatar
  • 1
0 votes
0 answers
44 views

Why does my loss function fluctuate so much?

I have a loss function that I'm trying to maximise using a neural network. While it does appear to increase and plateau over the training, it does so in a very "noisy" manner, spiking up and ...
VJ123's user avatar
  • 73
0 votes
0 answers
54 views

How to generate original training videos based on existing videoset?

I am a software engineer who is quickly ramping up on AI tech, but am nevertheless very new to the sector. A collegue has an extensive collection of training videos, the vertical is wheelchair seating ...
lukabloomrox's user avatar
0 votes
0 answers
43 views

Are autoencoders computationally cheaper than MLPs with the same number of neurons?

Are autoencoders computationally cheaper than other neural networks such as MLP with the same number of neurons? I have read in some papers that autoencoders train the network faster, and I could ...
Jesus M.'s user avatar
0 votes
1 answer
27 views

Do Quo et al (2013) perform backpropagation between layers?

Le et al. 2013's non-weight sharing CNN has inspired me to ask two questions on this site previously. When training the three-layer autoencoder, do they compute dL/dW (where L is equation 1) ...
Josiah Yoder's user avatar
0 votes
0 answers
65 views

What are the math theorems regarding the Multilayer Perceptron?

I've come across a theorem "Convergence theorem Simple Perceptron" for the first time, here-> https://zaguan.unizar.es/record/69205/files/TAZ-TFG-2018-148.pdf, page 27, (is in Spanish) ...
Verónica Rmz.'s user avatar
0 votes
0 answers
163 views

What is the difference between the forward pass of the Multi-Layer Perceptron, Deep AutoEncoder and Deep Belief Network?

Multi-Layer Perceptron (MLP), Deep AutoEncoder (DAE), and Deep Belief Network (DBN) are trained differently. However, do they follow the same process during the inference phase, i.e., do they ...
witdev's user avatar
  • 73
0 votes
0 answers
80 views

How to draw a 3-dimensonal shape's neural network

I am reading an exam question about NN (that I cannot publish, for copyright reasons). The question says: 'Construct a rectangle in 2D space. Define the lines, and then define the weights and ...
Slowat_Kela's user avatar