3

You can find a definition for "numerical stability" in mathworld wolframe: Numerical stability refers to how a malformed input affects the execution of an algorithm. In a numerically stable algorithm, errors in the input lessen in significance as the algorithm executes, having little effect on the final output. On the other hand, in a numerically ...


3

Second-order optimization algorithms like Hessian optimization have more information on the curvature of the loss function, so converge much, much faster than first-order optimization algorithms like gradient descent. I remember reading somewhere that if you have $n$ weights in the neural network, one iteration of a second-order optimization algorithm will ...


2

In some cases, you can solve a linear regression problem with an analytical (or closed-form) solution/expression (although this may not always be the best approach). See this answer for more details. Note that this solution involves matrix multiplications and the computation of an inverse with floating-point numbers, so this is still a numerical algorithm/...


1

Honourable mention: Memory-based approaches Although not analytic, memory-based models, such as k-nearest neighbours (k-NN) are very lightweight when learning, but have a higher cost to use the stored knowledge. Even though a k-NN model is slow to make inferences, the computation involved is not complex or iterative. It makes a single pass through all the ...


Only top voted, non community-wiki answers of a minimum length are eligible