Skip to main content
11 votes
Accepted

What is the time complexity of the forward pass algorithm of a feedforward neural network?

Let's suppose that we have an MLP with $15$ inputs, $20$ hidden neurons and $2$ output neurons. The operations performed are only in the hidden and output neurons, given that the input neurons only ...
nbro's user avatar
  • 40.9k
5 votes

What is the computational complexity of the forward pass of a convolutional neural network?

What is the time complexity? The time complexity of an algorithm is the number of basic operations, such as multiplications and summations, that the algorithm performs. The time complexity is usually ...
nbro's user avatar
  • 40.9k
3 votes

Why do language models produce different outputs for same prompt?

Language models produce a probability distribution over a set of words. You determine the next word by sampling from this distribution. So, determining the next word is stochastic even though the ...
SpiderRico's user avatar
  • 1,010
2 votes
Accepted

Are my computations of the forward and backward pass of a neural network with one input, hidden and output neurons correct?

One important point I missed in first review: error is a summatory, its derivative is also a summatory. About offsets "b": usually they are different in each cell (if not fixed to some value, as 0). ...
pasaba por aqui's user avatar
2 votes

How to compute the output of a neural network produced by NEAT?

The networks in NEAT are still implicitly layered. There are neurons that need to be evaluated before other neurons can be evaluated and so this gives us our layers. If you don't know the structure ...
Andrew Butler's user avatar
1 vote

What is the Preferred Mathematical Representation for a Forward Pass in a Neural Network?

I don't think there's a "standard way" of expressing the forward pass: you use the transpose when you need to use it, and this depends on how you define the weights and inputs matrices, and ...
nbro's user avatar
  • 40.9k

Only top scored, non community-wiki answers of a minimum length are eligible