# What kind of algorithm is the Levenberg–Marquardt algorithm?

Is a Levenberg–Marquardt algorithm a type of back-propagation algorithm or is it a different category of algorithm?

Wikipedia says that it is a curve fitting algorithm. How is a curve fitting algorithm relevant to a neural net?

## 1 Answer

In the context of Neural Networks, Backpropagation (with Gradient Descent, to use its full name) and Levengerg Marquardt are both members of the broader family of gradient descent algorithms. Backpropagation itself is not gradient descent, but it does the gradient climbing portion of a broader gradient descent algorithm.

You can imagine the function of a Neural Network as a function from its inputs to its outputs. If you were trying to solve, for example, a regression problem you can imagine this problem in multidimensional space with each training point corresponding to the coordinate provided by the values of its inputs and outputs (each of which represent a dimension). Then the entire training set you have for learning this regression problem become a set of points in this multidimensional space, and the function that your neural network performs is a curve in this multidimensional space. The closer this curve is to the points on the training set, the better it performs at the regression problem which essentially means we can generalize the task of training the neural network to a curve fitting problem.