Skip to main content

Questions tagged [forward-pass]

For questions about the "forward pass" algorithm of a neural network, i.e. the algorithm that transforms the input into the output of the neural network.

Filter by
Sorted by
Tagged with
0 votes
0 answers
21 views

Implement tensor operation with mini batches instead of Matrix multiplication in forward pass

The forward pass in neural network can be written as g(Wx+b). W is the weights Matrix, x is the input vector and b is bias, and g the non linearity, the activation function. However x can have more ...
0 votes
1 answer
451 views

How to compute the output of a neural network produced by NEAT?

I used to work with layered neural networks, where, given certain inputs, the output is produced layer-by-layer. With NEAT, a neural network may assume any topology, and they are no longer layered. So,...
3 votes
1 answer
487 views

Are my computations of the forward and backward pass of a neural network with one input, hidden and output neurons correct?

I have computed the forward and backward passes of the following simple neural network, with one input, hidden, and output neurons. Here are my computations of the forward pass. \begin{align} net_1 &...
2 votes
1 answer
2k views

Why do language models produce different outputs for same prompt?

For conventional 'Neural Networks', the weights simply act as a transformation in highly multi-dimensional space; for a forward pass, the output is always the same since there is no stochastic ...
1 vote
0 answers
23 views

Is there any existing mechanism that allows us to pass input from randomly selected layers of neural network per iteration?

Consider the following neural network with $\ell$ layers. $$i_0 \rightarrow h_1 \rightarrow h_2 \rightarrow h_3 \cdots \rightarrow h_{\ell-1} \rightarrow o_{\ell} ,$$ where $i, h, o$ stands for ...
0 votes
1 answer
10k views

What is the computational complexity of the forward pass of a convolutional neural network?

How do I determine the computational complexity (big-O notation) of the forward pass of a convolutional neural network? Let's assume for simplicity that we use zero-padding such that the input size ...
0 votes
0 answers
169 views

What is the difference between the forward pass of the Multi-Layer Perceptron, Deep AutoEncoder and Deep Belief Network?

Multi-Layer Perceptron (MLP), Deep AutoEncoder (DAE), and Deep Belief Network (DBN) are trained differently. However, do they follow the same process during the inference phase, i.e., do they ...
1 vote
0 answers
239 views

How can I perform the forward pass in a neural network evolved with NEAT, given that some connections may not exist or there may be loopy connections?

I have a problem that arose as part of a NEAT (Neuro Evolution Through Augmenting Topologies) implementation that I am writing. I am wanting it to produce topologies or graphs that describe neural ...
2 votes
0 answers
1k views

What is the time complexity of the forward pass and back-propagation of the sequence-to-sequence model with and without attention?

I keep looking through the literature, but can't seem to find any information regarding the time complexity of the forward pass and back-propagation of the sequence-to-sequence RNN encoder-decoder ...
2 votes
1 answer
153 views

What is the Preferred Mathematical Representation for a Forward Pass in a Neural Network?

I know this may be a question of semantics but I always see different articles explain forward pass slightly different. e.g. Sometimes they represent a forward pass to a hidden layer in a standard ...
11 votes
1 answer
7k views

What is the time complexity of the forward pass algorithm of a feedforward neural network?

How do I determine the time complexity of the forward pass algorithm of a feedforward neural network? How many multiplications are done to generate the output?
1 vote
0 answers
56 views

Could a quantum computer perform vectorized forward propagation in deep networks?

Forward propagation in Deep Neural Networks In the "Forward Propagation in a Deep Network" video on Coursera, Andrew NG mentions that there's no way to avoid a for loop to loop through the ...