Questions tagged [feedforward-neural-network]

For questions related to feedforward neural networks (FFNNs), which are also sometimes called multilayer perceptrons, but these two expressions may not always be interchangeable.

Filter by
Sorted by
Tagged with
0
votes
1answer
21 views

What are the most common feedforward neural networks?

What are the most common feedforward neural networks? What kind of inputs do they receive? For example, do they receive binary numbers, real numbers, vectors, or matrics? Is there such a taxonomy?
0
votes
0answers
16 views

How can feedforward neural networks act as contraction maps?

In graph neural networks, the Banach fixed-point theorem and Jacobi method it is described that the transition from one state to another be defined by a contraction map with a fixed-point. The autor ...
2
votes
1answer
35 views

Accuracy dropped when I ran the program the second time

I was following a tutorial about Feed Forward Networks and wrote this code for a simple FFN : ...
0
votes
0answers
19 views

Can neural networks always be assembled like Lego blocks?

BACKGROUND Consider a supervised problem which is based on two scalar features (1) and (2) as well as a third, "time-dependent", feature consisting of a sequence of five values (3)-(7). For ...
3
votes
2answers
101 views

What exactly is averaged when doing batch gradient descent?

I have a question about how the averaging works when doing mini-batch gradient descent. I think I now understood the general gradient descent algorithm, but only for online learning. When doing mini-...
2
votes
2answers
51 views

What are standard datasets for fully connected neural networks?

I am looking for datasets that are used as a testing standard in the fully connected neural networks (FCNN). For example, in the image recognition and CNN, CIFAR datasets are used in most of the ...
1
vote
0answers
20 views

How do we minimize loss for a single neuron with a feedback?

Suppose we had a series of single-dimensional data points $X = \{x_1, x_2, \dots, x_n \}$, where $n$ is the number of data points and there corresponding output values $T = \{t_1, t_2, \dots, t_n \}$. ...
4
votes
1answer
34 views

Can neurons in MLP and filters in CNN be compared?

I know they are not the same in working, but an input layer sends the input to x neurons with a set of weights, based of these weights and the activation layer, it produces an output that can be fed ...
1
vote
0answers
16 views

Can denoising auto-encoders be convolutional and fully connected?

I have been reading lately on autoencoders a lot. I just wanted to summarize my understanding of denoising autoencoders. As far as I understand they can be Fully connected (in which case, they will ...
3
votes
1answer
78 views

What is the simplest classification problem which cannot be solved by a perceptron?

What is the simplest classification problem which cannot be solved by a perceptron (that is a single-layered feed-forward neural network, with no hidden layers and step activation function), but it ...
1
vote
2answers
47 views

What happens if I train a network for more epochs, without using early stopping?

I have a question about training a neural network for more epochs even after the network has converged without using early stopping criterion. Consider the MNIST dataset and a LeNet 300-100-10 dense ...
1
vote
0answers
40 views

Why would my neural network run faster on my laptop than on my university's supercomputer?

I am trying to get my neural network running on my university's supercomputer in order to decrease its runtime (not for training, for testing - feedforward runs only). However, the Matlab function I ...
1
vote
1answer
47 views

Are there names for neural networks with a well-defined layer or neuron characteristics?

Are there names for neural networks with a well-defined layer or neuron characteristics? For example, a matrix that has the same number of rows and columns is called a square matrix. Is there an ...
1
vote
0answers
19 views

Neural network seems to just figure out the probability of a specific result

I am really new to neural networks, so i was following along with a video series, created by '3blue1brown' on youtube. I created an implementation of the network he explained in c++. I am attempting ...
2
votes
2answers
71 views

What is the name of this neural network architecture with layers that are also connected to non-neighbouring layers?

Consider a feedforward neural network. Suppose you have a layer of inputs, which is feedforward to a hidden layer, and feedforward both the input and hidden layers to an output layer. Is there a name ...
0
votes
1answer
31 views

How to train FFNN with Q-learning? [duplicate]

I know that in any NN architecture, the input data are states, and at the output layer Q-functionality of each action. Tell me please, how to adjust all weights in this case?
2
votes
0answers
27 views

How do I determine the best neural network architecture for a problem with 3 inputs and 12 outputs?

This post continues the topic in the following post: Is it possible to train a neural network with 3 inputs and 12 outputs?. I conducted several experiments in MATLAB and selected those neural ...
2
votes
1answer
60 views

How to make DNN learn multiplication/division?

A single neuron with 2 weights and identity activation can learn addition/subtraction as the 2 weights will converge to 1 and 1 (addition), or 1 and -1 (subtraction). However, for multiplication and ...
4
votes
0answers
38 views

Given an input $x \in R^{1\times d}$ and a network with $s$ hidden layers, is the time complexity of the forward pass $O(d^{2}s)$? [duplicate]

I have a neural network that takes as an input a vector of $x \in R^{1\times d}$ with $s$ hidden layers and each layer has $d$ neurons (including the output layer). If I understand correctly the ...
1
vote
0answers
24 views

Is this TensorFlow implementation of partial derivative of the cost with respect to the bias correct?

I have a neural network for MNIST classification which I am hard coding using TensorFlow 2.0. The neural network has an input layer consisting of 784 neurons (28 * 28), one hidden layer having "...
3
votes
1answer
59 views

What kind of data structures are needed to efficiently do back-propagation in a feedforward neural network?

In a feed-forward neural network, in order to efficiently do backpropagation, what kind of data structure is needed? I know the weights can just be stored in an array, and you need pointers of some ...
1
vote
0answers
15 views

Self-organizing map using weighted non-euclidean distance to minimize variance of predictions

Let's say I have a dataset, each item/row of which has $\mathit{X + 1}$ characteristics where the last characteristic (i.e., the $\mathit{1}$) represents the some value I want to predict, $\mathit{Y}$,...
8
votes
2answers
748 views

Why use a recurrent neural network over a feedforward neural network for sequence prediction?

If recurrent neural networks (RNNs) are used to capture prior information, couldn't the same thing be achieved by a feedforward neural network (FFNN) or multi-layer perceptron (MLP) where the inputs ...
1
vote
1answer
38 views

How can we print weights per iteration in a simple feed forward MLP for an specific class?

im working on a project in which I have to make a multi-layer perceptron with two hidden layers with 3 nodes in each. The target value in my data contains 8 unique values/classes. One of the tasks ...
1
vote
0answers
14 views

Matrix-output for FFNN?

Turns out that it looks like I will be approximating a 100x10 matrix in my project thesis. II have the following equation $y = Dx$, where $y$ is $(100 \times 1)$, $D$ is $100 \times 10$ and $x$ is $...
8
votes
2answers
986 views

Why would you implement the position-wise feed-forward network of the transformer with convolution layers?

The Transformer model introduced in "Attention is all you need" by Vaswani et al. incorporates a so-called position-wise feed-forward network (FFN): In addition to attention sub-layers, each of the ...
1
vote
0answers
29 views

How does a single neuron in hidden layer affect training accuracy

I'm currently a student learning about AI Networks. I've came across a statement in one of my Professor's books that a FFBP (Feed-Forward Back-Propagation) Neural Network with a single hidden layer ...
1
vote
1answer
45 views

Attempting to solve a optical character recognition task using a feed-forward network

I am doing some experimentation on neural networks, and for that I am trying to program a plain OCR task. I have learned CNNs are the best choice ,but for the time being and due to my inexperience, I ...
3
votes
0answers
49 views

Using a “is_padding” attribute in your padding instead of simply zero vectors

Typical Feed Forward Neural Networks require a fixed sized input and output. So when you have variable sized input, it seems to be common practice to pad the input with zero vectors. Why does it not ...
2
votes
1answer
72 views

Comparing and studying Loss Functions

I have a Deep Feedforward Neural Network $F: W \times \mathbb{R}^d \rightarrow \mathbb{R}^k$ (where $W$ is the space of the weights) with $L$ hidden layers, $m$ neurones per layer and ReLu activation. ...
6
votes
2answers
142 views

Neural network to detect “spam”?

I've inherited a neural network project at the company I work for. The person who developed gave me some very basic training to get up and running. I've maintained it for a while. The current neural ...
1
vote
1answer
425 views

Feed forward neural network using numpy for IRIS dataset

I tried to build a neural network for working on IRIS dataset using only numpy after reading an article (link: https://iamtrask.github.io/2015/07/12/basic-python-network/). I tried to search the ...
2
votes
1answer
150 views

Maximum number of neurons in a layer given number of neurons in previous layer

Consider an extremely complicated feed-forward neural network training example but with no need of computational efficiency or limiting of processing time. What is the maximum number of hidden ...
2
votes
1answer
168 views

Significance of depth of a deep neural network

How is a feed-forward neural network with few hidden layers and lots of nodes in those hidden layers different from a network with a lot of hidden layers but relatively lesser nodes in those hidden ...
2
votes
2answers
905 views

Should the weights of a neural network be updated after each example or at the end of the batch? [duplicate]

Should the weights of a neural network be updated after each example or at the end of the batch? Do I need a normalization factor in the second case?
3
votes
1answer
57 views

Learning an arbitrary function using a feedforward net

I would like to get a simple example running in matlab that will use a neural net to learn an arbitrary function from input output data (basically model identification) and then be able to approximate ...
1
vote
3answers
499 views

How to create Partially Connected NNs with prespecified connections using Tensorflow?

I'd like to implement a partially connected neural network with ~3 to 4 hidden layers (a sparse deep neural network?) where I can specify which node connects to which node from the previous/next layer....
1
vote
1answer
64 views

predict waste generation

I am starting a project to predict the generation of urban waste. I have found very little information on this topic on the internet. I would be very useful advice on how to approach this topic, and ...
5
votes
1answer
52 views

Which marketing-related classification challenges is a feed forward neural network suited to solve?

I am trying to think of some marketing-related classification challenges that a feed-forward neural network would be suited for. Any ideas?
1
vote
1answer
551 views

Neural Network on EV3 Mindstorm without 3rd Party Software

I am working on a prototype for an Ev3 Neural Network. Because for competitions, we are not allowed to use Bluetooth or Wifi connections, the neural network must be made with the Ev3 block-based ...
2
votes
1answer
822 views

Compute Jacobian matrix of Deep learning model?

I am trying to implement this paper. In this paper, the author uses the forward derivative to compute the Jacobian matrix dF/dx using chain rule where F is the probability got from the last layer and ...
6
votes
2answers
136 views

Evolving network in game

So I wrote simple feed forward neural network that plays tic-tac-toe: 9 neurons in input layers: 1 - my sign, -1 - opponent's sign, 0 - empty; 9 neurons in hidden layer: value calculated using Relu; ...
2
votes
1answer
75 views

Are there benchmarks for assessing the speed of the forward-pass of neural networks?

I have a task where I would like to use a convolutional neural network (CNN). I would like to incrementally start from the fastest models, fine-tune and see whether they fit my "budget". At the moment,...
2
votes
1answer
153 views

What is the difference between a feed-forward neural network and a liquid state machine?

I have used a FFNN and LSM to perform the same task, namely, to predict the sentence "How are you". The LSM gave me more accurate results than FFNN. However, the LSM did not produce perfect prediction ...
5
votes
1answer
478 views

Why aren't there neural networks that connect the output of each layer to all next layers?

Why aren't there neural networks that connect the output of each layer to all next layers? For example, the output of layer 1 would be fed to the input of layers 2, 3, 4, etc. Beyond computational ...
14
votes
2answers
259 views

How do I decide the optimal number of layers for a neural network?

How do I decide the optimal number of layers for a neural network (feedforward or recurrent)?
4
votes
1answer
117 views

What is the significance of weights in a feedforward neural network?

In a feedforward neural network the inputs are fed directly to the outputs via a series of weights. What purpose do the weights serve and how are they significant in this neural network?