Questions tagged [feedforward-neural-network]

For questions related to feedforward neural networks (FFNNs), which are also sometimes called multilayer perceptrons, but these two expressions may not always be interchangeable.

Filter by
Sorted by
Tagged with
4
votes
0answers
31 views

Given an input $x \in R^{1\times d}$ and a network with $s$ hidden layers, is the time complexity of the forward pass $O(d^{2}s)$? [duplicate]

I have a neural network that takes as an input a vector of $x \in R^{1\times d}$ with $s$ hidden layers and each layer has $d$ neurons (including the output layer). If I understand correctly the ...
1
vote
0answers
19 views

Is this TensorFlow implementation of partial derivative of the cost with respect to the bias correct?

I have a neural network for MNIST classification which I am hard coding using TensorFlow 2.0. The neural network has an input layer consisting of 784 neurons (28 * 28), one hidden layer having "...
3
votes
1answer
45 views

What kind of data structures are needed to efficiently do back-propagation in a feedforward neural network?

In a feed-forward neural network, in order to efficiently do backpropagation, what kind of data structure is needed? I know the weights can just be stored in an array, and you need pointers of some ...
1
vote
0answers
14 views

Self-organizing map using weighted non-euclidean distance to minimize variance of predictions

Let's say I have a dataset, each item/row of which has $\mathit{X + 1}$ characteristics where the last characteristic (i.e., the $\mathit{1}$) represents the some value I want to predict, $\mathit{Y}$,...
7
votes
2answers
630 views

Why use a recurrent neural network over a feedforward neural network for sequence prediction?

If recurrent neural networks (RNNs) are used to capture prior information, couldn't the same thing be achieved by a feedforward neural network (FFNN) or multi-layer perceptron (MLP) where the inputs ...
0
votes
1answer
25 views

How can we print weights per iteration in a simple feed forward MLP for an specific class?

im working on a project in which I have to make a multi-layer perceptron with two hidden layers with 3 nodes in each. The target value in my data contains 8 unique values/classes. One of the tasks ...
1
vote
0answers
11 views

Matrix-output for FFNN?

Turns out that it looks like I will be approximating a 100x10 matrix in my project thesis. II have the following equation $y = Dx$, where $y$ is $(100 \times 1)$, $D$ is $100 \times 10$ and $x$ is $...
6
votes
0answers
117 views

Why would you implement the position-wise feed-forward network of the transformer with convolution layers?

The Transformer model introduced in "Attention is all you need" by Vaswani et al. incorporates a so-called position-wise feed-forward network (FFN): In addition to attention sub-layers, each of the ...
1
vote
0answers
28 views

How does a single neuron in hidden layer affect training accuracy

I'm currently a student learning about AI Networks. I've came across a statement in one of my Professor's books that a FFBP (Feed-Forward Back-Propagation) Neural Network with a single hidden layer ...
0
votes
0answers
13 views

How is the length of an input sequence related to the structure of an RNN?

My question is only with regards to the feedforward part of an RNN. I am following these steps. I am working on prediction of a time series. The time series is a toy model generated by me. It is ...
0
votes
1answer
40 views

Attempting to solve a optical character recognition task using a feed-forward network

I am doing some experimentation on neural networks, and for that I am trying to program a plain OCR task. I have learned CNNs are the best choice ,but for the time being and due to my inexperience, I ...
3
votes
0answers
48 views

Using a “is_padding” attribute in your padding instead of simply zero vectors

Typical Feed Forward Neural Networks require a fixed sized input and output. So when you have variable sized input, it seems to be common practice to pad the input with zero vectors. Why does it not ...
2
votes
1answer
61 views

Comparing and studying Loss Functions

I have a Deep Feedforward Neural Network $F: W \times \mathbb{R}^d \rightarrow \mathbb{R}^k$ (where $W$ is the space of the weights) with $L$ hidden layers, $m$ neurones per layer and ReLu activation. ...
5
votes
2answers
122 views

Neural network to detect “spam”?

I've inherited a neural network project at the company I work for. The person who developed gave me some very basic training to get up and running. I've maintained it for a while. The current neural ...
0
votes
1answer
296 views

Feed forward neural network using numpy for IRIS dataset

I tried to build a neural network for working on IRIS dataset using only numpy after reading an article (link: https://iamtrask.github.io/2015/07/12/basic-python-network/). I tried to search the ...
1
vote
1answer
96 views

Maximum number of neurons in a layer given number of neurons in previous layer

Consider an extremely complicated feed-forward neural network training example but with no need of computational efficiency or limiting of processing time. What is the maximum number of hidden ...
2
votes
1answer
143 views

Significance of depth of a deep neural network

How is a feed-forward neural network with few hidden layers and lots of nodes in those hidden layers different from a network with a lot of hidden layers but relatively lesser nodes in those hidden ...
2
votes
2answers
725 views

Should the weights of a neural network be updated after each example or at the end of the batch? [duplicate]

Should the weights of a neural network be updated after each example or at the end of the batch? Do I need a normalization factor in the second case?
3
votes
1answer
46 views

Learning an arbitrary function using a feedforward net

I would like to get a simple example running in matlab that will use a neural net to learn an arbitrary function from input output data (basically model identification) and then be able to approximate ...
1
vote
3answers
357 views

How to create Partially Connected NNs with prespecified connections using Tensorflow?

I'd like to implement a partially connected neural network with ~3 to 4 hidden layers (a sparse deep neural network?) where I can specify which node connects to which node from the previous/next layer....
1
vote
1answer
62 views

predict waste generation

I am starting a project to predict the generation of urban waste. I have found very little information on this topic on the internet. I would be very useful advice on how to approach this topic, and ...
5
votes
1answer
52 views

Which marketing-related classification challenges is a feed forward neural network suited to solve?

I am trying to think of some marketing-related classification challenges that a feed-forward neural network would be suited for. Any ideas?
1
vote
1answer
407 views

Neural Network on EV3 Mindstorm without 3rd Party Software

I am working on a prototype for an Ev3 Neural Network. Because for competitions, we are not allowed to use Bluetooth or Wifi connections, the neural network must be made with the Ev3 block-based ...
2
votes
1answer
669 views

Compute Jacobian matrix of Deep learning model?

I am trying to implement this paper. In this paper, the author uses the forward derivative to compute the Jacobian matrix dF/dx using chain rule where F is the probability got from the last layer and ...
6
votes
2answers
131 views

Evolving network in game

So I wrote simple feed forward neural network that plays tic-tac-toe: 9 neurons in input layers: 1 - my sign, -1 - opponent's sign, 0 - empty; 9 neurons in hidden layer: value calculated using Relu; ...
2
votes
1answer
57 views

Are there benchmarks for assessing the speed of the forward-pass of neural networks?

I have a task where I would like to use a convolutional neural network (CNN). I would like to incrementally start from the fastest models, fine-tune and see whether they fit my "budget". At the moment,...
2
votes
1answer
138 views

What is the difference between a feed-forward neural network and a liquid state machine?

I have used a FFNN and LSM to perform the same task, namely, to predict the sentence "How are you". The LSM gave me more accurate results than FFNN. However, the LSM did not produce perfect prediction ...
4
votes
1answer
104 views

What is the significance of weights in a feedforward neural network?

In a feedforward neural network the inputs are fed directly to the outputs via a series of weights. What purpose do the weights serve and how are they significant in this neural network?