8
$\begingroup$

What does "backprop" mean? I've Googled it, but it's showing backpropagation.

Is the "backprop" term basically the same as "backpropagation" or does it have a different meaning?

$\endgroup$
  • 1
    $\begingroup$ Same. It is used to tune the weights of the neural network. $\endgroup$ – user Aug 22 '16 at 23:32
  • $\begingroup$ just to make sure I understood it. You googled it, have not found anything, asked a question here, got many the same answers and posted your own answer with better googling that restates the same thing as everyone else told you a day before. Am I right? $\endgroup$ – Salvador Dali Sep 18 '16 at 23:43
  • $\begingroup$ @SalvadorDali Correct, I've asked it, because I was confused whether it means the same. People answered with one/few sentences without giving any references saying it's the same. This was not enough for me, so I've decided to answer my-self by giving some extra references to backup the claims and explain that it's actually a short for 'backpropagation of error', which was not mentioned before. This was the 1st question of the site btw. $\endgroup$ – kenorb Sep 19 '16 at 0:42
12
$\begingroup$

"Backprop" is the same as "backpropagation": it's just a shorter way to say it. It is sometimes abbreviated as "BP".

$\endgroup$
7
$\begingroup$

'Backprop' is short for 'backpropagation of error' in order to avoid confusion when using backpropagation term.

Basically backpropagation refers to the method for computing the gradient of the case-wise error function with respect to the weights for a feedforward networkWerbos. And backprop refers to a training method that uses backpropagation to compute the gradient.

So we can say that a backprop network is a feedforward network trained by backpropagation.

The 'standard backprop' term is a euphemism for the generalized delta rule which is most widely used supervised training method.

Source: What is backprop? at FAQ of Usenet newsgroup comp.ai.neural-nets

References:

  • Werbos, P. J. (1974). Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. PhD thesis, Harvard University.
  • Werbos, P. J. (1994). The Roots of Backpropagation: From Ordered Derivatives to Neural Networks and Political Forecasting,Wiley Interscience.
  • Bertsekas, D. P. (1995), Nonlinear Programming, Belmont, MA: Athena Scientific, ISBN 1-886529-14-0.
  • Bertsekas, D. P. and Tsitsiklis, J. N. (1996), Neuro-Dynamic Programming, Belmont, MA: Athena Scientific, ISBN 1-886529-10-8.
  • Polyak, B.T. (1964), "Some methods of speeding up the convergence of iteration methods," Z. Vycisl. Mat. i Mat. Fiz., 4, 1-17.
  • Polyak, B.T. (1987), Introduction to Optimization, NY: Optimization Software, Inc.
  • Reed, R.D., and Marks, R.J, II (1999), Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks, Cambridge, MA: The MIT Press, ISBN 0-262-18190-8.
  • Rumelhart, D.E., Hinton, G.E., and Williams, R.J. (1986), "Learning internal representations by error propagation", in Rumelhart, D.E. and McClelland, J. L., eds. (1986), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 1, 318-362, Cambridge, MA: The MIT Press.
  • Werbos, P.J. (1974/1994), The Roots of Backpropagation, NY: John Wiley & Sons. Includes Werbos's 1974 Harvard Ph.D. thesis, Beyond Regression.
$\endgroup$
  • 2
    $\begingroup$ Nice detailed answer with references for further reading +1 ! $\endgroup$ – Tshilidzi Mudau Jun 7 '17 at 7:13
2
$\begingroup$

Yes, as Franck has rightly put, "backprop" means backpropogation, which is frequently used in the domain of neural networks for error optimization.

For a detailed explanation, I would point out this tutorial on the concept of backpropogation by a very good book of Michael Nielsen.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.