Questions tagged [forward-pass]

For questions about the "forward pass" algorithm of a neural network, i.e. the algorithm that transforms the input into the output of the neural network.

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

Can someone give me an example that shows the working of Vector mode Forward Automatic Differentiation?

Given a Function F(x,y,z), and I want to calculate the derivative of the function with respect to x,y and z, forward mode generally will take 3 passes to compute the derivative, one pass for each x,y ...
user avatar
0 votes
1 answer
68 views

Why do language models produce different outputs for same prompt?

For conventional 'Neural Networks', the weights simply act as a transformation in highly multi-dimensional space; for a forward pass, the output is always the same since there is no stochastic ...
user avatar
  • 126
1 vote
0 answers
21 views

Is there any existing mechanism that allows us to pass input from randomly selected layers of neural network per iteration?

Consider the following neural network with $\ell$ layers. $$i_0 \rightarrow h_1 \rightarrow h_2 \rightarrow h_3 \cdots \rightarrow h_{\ell-1} \rightarrow o_{\ell} ,$$ where $i, h, o$ stands for ...
user avatar
  • 3,201
0 votes
0 answers
29 views

Are there any good references that describe the equations of the forward pass of Graph Neural Networks?

I am trying to program Graph Neural Network from scratch. Can the community please suggest a good reference/s to read about the equations of the forward pass in Graph Neural Networks, especially in ...
user avatar
  • 121
0 votes
0 answers
130 views

What is the difference between the forward pass of the Multi-Layer Perceptron, Deep AutoEncoder and Deep Belief Network?

Multi-Layer Perceptron (MLP), Deep AutoEncoder (DAE), and Deep Belief Network (DBN) are trained differently. However, do they follow the same process during the inference phase, i.e., do they ...
user avatar
  • 73
1 vote
0 answers
144 views

How can I perform the forward pass in a neural network evolved with NEAT, given that some connections may not exist or there may be loopy connections?

I have a problem that arose as part of a NEAT (Neuro Evolution Through Augmenting Topologies) implementation that I am writing. I am wanting it to produce topologies or graphs that describe neural ...
user avatar
2 votes
1 answer
92 views

What is the Preferred Mathematical Representation for a Forward Pass in a Neural Network?

I know this may be a question of semantics but I always see different articles explain forward pass slightly different. e.g. Sometimes they represent a forward pass to a hidden layer in a standard ...
user avatar
0 votes
1 answer
6k views

What is the computational complexity of the forward pass of a convolutional neural network?

How do I determine the computational complexity (big-O notation) of the forward pass of a convolutional neural network? Let's assume for simplicity that we use zero-padding such that the input size ...
user avatar
  • 9
1 vote
0 answers
34 views

Could a quantum computer perform vectorized forward propagation in deep networks?

Forward propagation in Deep Neural Networks In the "Forward Propagation in a Deep Network" video on Coursera, Andrew NG mentions that there's no way to avoid a for loop to loop through the ...
user avatar
  • 111
2 votes
0 answers
819 views

What is the time complexity of the forward pass and back-propagation of the sequence-to-sequence model with and without attention?

I keep looking through the literature, but can't seem to find any information regarding the time complexity of the forward pass and back-propagation of the sequence-to-sequence RNN encoder-decoder ...
user avatar
9 votes
1 answer
4k views

What is the time complexity of the forward pass algorithm of a feedforward neural network?

How do I determine the time complexity of the forward pass algorithm of a feedforward neural network? How many multiplications are done to generate the output?
user avatar
3 votes
1 answer
368 views

Are my computations of the forward and backward pass of a neural network with one input, hidden and output neurons correct?

I have computed the forward and backward passes of the following simple neural network, with one input, hidden, and output neurons. Here are my computations of the forward pass. \begin{align} net_1 &...
user avatar
  • 946
0 votes
1 answer
297 views

How to compute the output of a neural network produced by NEAT?

I used to work with layered neural networks, where, given certain inputs, the output is produced layer-by-layer. With NEAT, a neural network may assume any topology, and they are no longer layered. So,...
user avatar
  • 331