Questions tagged [feedforward-neural-networks]

For questions related to feedforward neural networks (FFNNs), which are also sometimes called multilayer perceptrons, but these two expressions may not always be interchangeable.

27 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3 votes
0 answers
615 views

How to use residual learning applied to fully connected networks?

Is there any reason why skip connections would not provide the same benefits to fully connected layers as it does for convolutional? I've read the ResNet paper and it says that the applications should ...
rocksNwaves's user avatar
3 votes
1 answer
119 views

Attempting to solve a optical character recognition task using a feed-forward network

I am doing some experimentation on neural networks, and for that I am trying to program a plain OCR task. I have learned CNNs are the best choice ,but for the time being and due to my inexperience, I ...
Chal.lo's user avatar
  • 51
3 votes
0 answers
58 views

Using a "is_padding" attribute in your padding instead of simply zero vectors

Typical Feed Forward Neural Networks require a fixed sized input and output. So when you have variable sized input, it seems to be common practice to pad the input with zero vectors. Why does it not ...
NateW's user avatar
  • 153
3 votes
1 answer
64 views

Which marketing-related classification challenges is a feed forward neural network suited to solve?

I am trying to think of some marketing-related classification challenges that a feed-forward neural network would be suited for. Any ideas?
Ryan Cameron's user avatar
2 votes
1 answer
161 views

Closed networks vs Networks with a removed delay to predict new data

I've come across two types of neural networks to predict, both from Matlab, the closed structure and the net that removes one delay to find new data. From Matlab's app generated scripts we see: % ...
Verónica Rmz.'s user avatar
2 votes
0 answers
220 views

Bias gradient of layer before batch normalization always zero

From the original paper and this post we have that batch normalization backpropagation can be formulated as I'm interested in the derivative of the previous layer outputs $x_i=\sigma(w X_i+b)$ with ...
Matthias's user avatar
2 votes
0 answers
289 views

Why should variance(output) equal variance(input) in Xavier Initialisation?

In a lot of explanations online for Xavier Initialization, I see the following: With each passing layer, we want the variance to remain the same. This helps us keep the signal from exploding to a ...
THAT_AI_GUY's user avatar
2 votes
0 answers
43 views

How do I determine the best neural network architecture for a problem with 3 inputs and 12 outputs?

This post continues the topic in the following post: Is it possible to train a neural network with 3 inputs and 12 outputs?. I conducted several experiments in MATLAB and selected those neural ...
dtn's user avatar
  • 239
1 vote
0 answers
39 views

Is there a neural network architecture specialized for mapping lower-to-higher-dimensional data?

I am building a neural network that takes in a set of 86 parameters (primarily architecture-related, such as building floor area, kitchen size, number of a certain type of furniture, etc.) and outputs ...
JS4137's user avatar
  • 143
1 vote
1 answer
83 views

How does a multidimensional vector get fed into a single node in a neural network?

I mostly develop neural networks completely from scratch, like without libraries. I've been seeing, especially in NLP tasks, entire vectors, often representing words, get fed into a single node. I'm ...
Jake StBu's user avatar
1 vote
0 answers
152 views

How can an MLP be implemented with convolutional layers?

I am studying the architecture of the network pointnet, specifically the MLPs stages of the pipeline highlighted in red in the following image (taken from the author page here): It is strange to find ...
Jacob Morales Gonzalez's user avatar
1 vote
0 answers
59 views

How to code mathematical functions in terms of neural networks?

I'm exploring how to solve differential equations using neural networks and discovered this Lagaris et. al. paper. In the paper, the solution to a differential equation is written as a mathematical ...
mombash's user avatar
  • 11
1 vote
0 answers
50 views

Can I help my neural network if I know the sign of the relationships between inputs and outputs

I am attempting to train a neural network where I can say the following: For most inputs, I know the sign of the relationship between that input and several specific outputs. I.e. whatever set of ...
Mick's user avatar
  • 111
1 vote
0 answers
131 views

Does there exist functions for which the necessary number of nodes in a shallow neural network tends to infinity as approximation error tends to 0?

The Universal Approximation Theorem states (roughly) that any continuous function can be approximated to within an arbitrary precision $\varepsilon>0$ by a feedforward neural network with one ...
GraftCraft's user avatar
1 vote
0 answers
57 views

ReLU function converging to local optimum in one case and diverging in the other one

I implemented a simple neural network with 1 hidden layer. I used ReLU as activation function for the hidden layer and the output layer just uses the linear function. To check my implementation I ...
SAGALPREET SINGH's user avatar
1 vote
0 answers
38 views

What's the difference between RNNs and Feed Forward Neural Networks if a fixed size vector can preserve sequential information?

I was watching a Youtube video in which the problem of trying to predict the last word in a sentence was posed. The sentence was "I took my cat for a" and the last word was "walk"....
Prabhav Arora's user avatar
1 vote
0 answers
47 views

How do we minimize loss for a single neuron with a feedback?

Suppose we had a series of single-dimensional data points $X = \{x_1, x_2, \dots, x_n \}$, where $n$ is the number of data points and there corresponding output values $T = \{t_1, t_2, \dots, t_n \}$. ...
Manish Kausik Hari Baskar's user avatar
1 vote
0 answers
32 views

Can denoising auto-encoders be convolutional and fully connected?

I have been reading lately on autoencoders a lot. I just wanted to summarize my understanding of denoising autoencoders. As far as I understand they can be Fully connected (in which case, they will ...
PagMax's user avatar
  • 111
1 vote
0 answers
30 views

Neural network seems to just figure out the probability of a specific result

I am really new to neural networks, so i was following along with a video series, created by '3blue1brown' on youtube. I created an implementation of the network he explained in c++. I am attempting ...
finlay morrison's user avatar
1 vote
0 answers
173 views

Is this TensorFlow implementation of partial derivative of the cost with respect to the bias correct?

I have a neural network for MNIST classification which I am hard coding using TensorFlow 2.0. The neural network has an input layer consisting of 784 neurons (28 * 28), one hidden layer having "...
Arun's user avatar
  • 235
1 vote
0 answers
21 views

Self-organizing map using weighted non-euclidean distance to minimize variance of predictions

Let's say I have a dataset, each item/row of which has $\mathit{X + 1}$ characteristics where the last characteristic (i.e., the $\mathit{1}$) represents the some value I want to predict, $\mathit{Y}$,...
SuperCodeBrah's user avatar
1 vote
0 answers
26 views

Matrix-output for FFNN?

Turns out that it looks like I will be approximating a 100x10 matrix in my project thesis. II have the following equation $y = Dx$, where $y$ is $(100 \times 1)$, $D$ is $100 \times 10$ and $x$ is $...
hakver29's user avatar
1 vote
2 answers
1k views

How to create Partially Connected NNs with prespecified connections using Tensorflow?

I'd like to implement a partially connected neural network with ~3 to 4 hidden layers (a sparse deep neural network?) where I can specify which node connects to which node from the previous/next layer....
Pınar Demetci's user avatar
0 votes
0 answers
48 views

Why does my loss function fluctuate so much?

I have a loss function that I'm trying to maximise using a neural network. While it does appear to increase and plateau over the training, it does so in a very "noisy" manner, spiking up and ...
VJ123's user avatar
  • 73
0 votes
0 answers
12 views

How are groups created in maxout units when dividing the set of inputs 𝑧 into groups of 𝑘 values?

I don't get $G^(i)$the set of indices into the inputs for group $i$, $\{(i −1)k+ 1, . . . , ik\}$ when creating a maxout units/function, these thing that outputs the maximum element of groups: $$g(z)...
Revolucion for Monica's user avatar
0 votes
0 answers
53 views

Backpropagation of position-wise feedforward neural network

I have read a paper entitled "Attention is all you need" by Vaswani et al. (2017). This paper use the so-called position-wise feedforward neural network, where the input of this network is a ...
poglhar's user avatar
  • 23
0 votes
2 answers
788 views

Why isn't the loss of my neural network reduced after 2500 iterations?

I have developed a basic feedforward neural network from scratch to classify whether image is of cat or not cat. It works fine, but after 2500 iterations, my cost function is not reducing properly. ...
Siddarth's user avatar