Questions tagged [forward-pass]
For questions about the "forward pass" algorithm of a neural network, i.e. the algorithm that transforms the input into the output of the neural network.
12
questions
0
votes
0
answers
15
views
Can someone give me an example that shows the working of Vector mode Forward Automatic Differentiation?
Given a Function F(x,y,z), and I want to calculate the derivative of the function with respect to x,y and z, forward mode generally will take 3 passes to compute the derivative, one pass for each x,y ...
0
votes
1
answer
332
views
Why do language models produce different outputs for same prompt?
For conventional 'Neural Networks', the weights simply act as a transformation in highly multi-dimensional space; for a forward pass, the output is always the same since there is no stochastic ...
- 126
1
vote
0
answers
22
views
Is there any existing mechanism that allows us to pass input from randomly selected layers of neural network per iteration?
Consider the following neural network with $\ell$ layers.
$$i_0 \rightarrow h_1 \rightarrow h_2 \rightarrow h_3 \cdots \rightarrow h_{\ell-1} \rightarrow o_{\ell} ,$$
where $i, h, o$ stands for ...
- 3,481
0
votes
0
answers
148
views
What is the difference between the forward pass of the Multi-Layer Perceptron, Deep AutoEncoder and Deep Belief Network?
Multi-Layer Perceptron (MLP), Deep AutoEncoder (DAE), and Deep Belief Network (DBN) are trained differently.
However, do they follow the same process during the inference phase, i.e., do they ...
- 73
1
vote
0
answers
185
views
How can I perform the forward pass in a neural network evolved with NEAT, given that some connections may not exist or there may be loopy connections?
I have a problem that arose as part of a NEAT (Neuro Evolution Through Augmenting Topologies) implementation that I am writing. I am wanting it to produce topologies or graphs that describe neural ...
- 106
2
votes
1
answer
111
views
What is the Preferred Mathematical Representation for a Forward Pass in a Neural Network?
I know this may be a question of semantics but I always see different articles explain forward pass slightly different. e.g. Sometimes they represent a forward pass to a hidden layer in a standard ...
- 121
0
votes
1
answer
8k
views
What is the computational complexity of the forward pass of a convolutional neural network?
How do I determine the computational complexity (big-O notation) of the forward pass of a convolutional neural network?
Let's assume for simplicity that we use zero-padding such that the input size ...
- 9
1
vote
0
answers
38
views
Could a quantum computer perform vectorized forward propagation in deep networks?
Forward propagation in Deep Neural Networks
In the "Forward Propagation in a Deep Network" video on Coursera, Andrew NG mentions that there's no way to avoid a for loop to loop through the ...
- 111
2
votes
0
answers
1k
views
What is the time complexity of the forward pass and back-propagation of the sequence-to-sequence model with and without attention?
I keep looking through the literature, but can't seem to find any information regarding the time complexity of the forward pass and back-propagation of the sequence-to-sequence RNN encoder-decoder ...
- 53
10
votes
1
answer
5k
views
What is the time complexity of the forward pass algorithm of a feedforward neural network?
How do I determine the time complexity of the forward pass algorithm of a feedforward neural network? How many multiplications are done to generate the output?
- 103
3
votes
1
answer
445
views
Are my computations of the forward and backward pass of a neural network with one input, hidden and output neurons correct?
I have computed the forward and backward passes of the following simple neural network, with one input, hidden, and output neurons.
Here are my computations of the forward pass.
\begin{align}
net_1 &...
- 1,006
0
votes
1
answer
367
views
How to compute the output of a neural network produced by NEAT?
I used to work with layered neural networks, where, given certain inputs, the output is produced layer-by-layer.
With NEAT, a neural network may assume any topology, and they are no longer layered. So,...
- 331