11
votes
Accepted
What is the time complexity of the forward pass algorithm of a feedforward neural network?
Let's suppose that we have an MLP with $15$ inputs, $20$ hidden neurons and $2$ output neurons. The operations performed are only in the hidden and output neurons, given that the input neurons only ...
- 39.6k
5
votes
What is the computational complexity of the forward pass of a convolutional neural network?
What is the time complexity?
The time complexity of an algorithm is the number of basic operations, such as multiplications and summations, that the algorithm performs. The time complexity is usually ...
- 39.6k
3
votes
Why do language models produce different outputs for same prompt?
Language models produce a probability distribution over a set of words. You determine the next word by sampling from this distribution. So, determining the next word is stochastic even though the ...
- 980
2
votes
Accepted
Are my computations of the forward and backward pass of a neural network with one input, hidden and output neurons correct?
One important point I missed in first review: error is a summatory, its derivative is also a summatory.
About offsets "b": usually they are different in each cell (if not fixed to some value, as 0). ...
- 1,283
2
votes
How to compute the output of a neural network produced by NEAT?
The networks in NEAT are still implicitly layered. There are neurons that need to be evaluated before other neurons can be evaluated and so this gives us our layers.
If you don't know the structure ...
- 587
1
vote
What is the Preferred Mathematical Representation for a Forward Pass in a Neural Network?
I don't think there's a "standard way" of expressing the forward pass: you use the transpose when you need to use it, and this depends on how you define the weights and inputs matrices, and ...
- 39.6k
Only top scored, non community-wiki answers of a minimum length are eligible