Questions tagged [forward-pass]
For questions about the "forward pass" algorithm of a neural network, i.e. the algorithm that transforms the input into the output of the neural network.
12
questions
0
votes
0
answers
21
views
Implement tensor operation with mini batches instead of Matrix multiplication in forward pass
The forward pass in neural network can be written as g(Wx+b). W is the weights Matrix, x is the input vector and b is bias, and g the non linearity, the activation function. However x can have more ...
- 301
3
votes
1
answer
3k
views
Why do language models produce different outputs for same prompt?
For conventional 'Neural Networks', the weights simply act as a transformation in highly multi-dimensional space; for a forward pass, the output is always the same since there is no stochastic ...
- 156
1
vote
0
answers
23
views
Is there any existing mechanism that allows us to pass input from randomly selected layers of neural network per iteration?
Consider the following neural network with $\ell$ layers.
$$i_0 \rightarrow h_1 \rightarrow h_2 \rightarrow h_3 \cdots \rightarrow h_{\ell-1} \rightarrow o_{\ell} ,$$
where $i, h, o$ stands for ...
- 3,950
0
votes
0
answers
170
views
What is the difference between the forward pass of the Multi-Layer Perceptron, Deep AutoEncoder and Deep Belief Network?
Multi-Layer Perceptron (MLP), Deep AutoEncoder (DAE), and Deep Belief Network (DBN) are trained differently.
However, do they follow the same process during the inference phase, i.e., do they ...
- 73
1
vote
0
answers
251
views
How can I perform the forward pass in a neural network evolved with NEAT, given that some connections may not exist or there may be loopy connections?
I have a problem that arose as part of a NEAT (Neuro Evolution Through Augmenting Topologies) implementation that I am writing. I am wanting it to produce topologies or graphs that describe neural ...
- 106
2
votes
1
answer
158
views
What is the Preferred Mathematical Representation for a Forward Pass in a Neural Network?
I know this may be a question of semantics but I always see different articles explain forward pass slightly different. e.g. Sometimes they represent a forward pass to a hidden layer in a standard ...
- 121
0
votes
1
answer
10k
views
What is the computational complexity of the forward pass of a convolutional neural network?
How do I determine the computational complexity (big-O notation) of the forward pass of a convolutional neural network?
Let's assume for simplicity that we use zero-padding such that the input size ...
- 9
1
vote
0
answers
61
views
Could a quantum computer perform vectorized forward propagation in deep networks?
Forward propagation in Deep Neural Networks
In the "Forward Propagation in a Deep Network" video on Coursera, Andrew NG mentions that there's no way to avoid a for loop to loop through the ...
- 111
2
votes
0
answers
1k
views
What is the time complexity of the forward pass and back-propagation of the sequence-to-sequence model with and without attention?
I keep looking through the literature, but can't seem to find any information regarding the time complexity of the forward pass and back-propagation of the sequence-to-sequence RNN encoder-decoder ...
- 53
11
votes
1
answer
8k
views
What is the time complexity of the forward pass algorithm of a feedforward neural network?
How do I determine the time complexity of the forward pass algorithm of a feedforward neural network? How many multiplications are done to generate the output?
- 113
3
votes
1
answer
491
views
Are my computations of the forward and backward pass of a neural network with one input, hidden and output neurons correct?
I have computed the forward and backward passes of the following simple neural network, with one input, hidden, and output neurons.
Here are my computations of the forward pass.
\begin{align}
net_1 &...
- 1,076
0
votes
1
answer
456
views
How to compute the output of a neural network produced by NEAT?
I used to work with layered neural networks, where, given certain inputs, the output is produced layer-by-layer.
With NEAT, a neural network may assume any topology, and they are no longer layered. So,...
- 341