4
votes
Accepted
What is numerical stability?
You can find a definition for "numerical stability" in mathworld wolframe:
Numerical stability refers to how a malformed input affects the execution of an algorithm. In a numerically stable ...
- 1,723
4
votes
Accepted
Why is second-order backpropagation useful?
Second-order optimization algorithms like Hessian optimization have more information on the curvature of the loss function, so converge much, much faster than first-order optimization algorithms like ...
- 1,400
2
votes
What are most commons methods to measure improvement rate in a meta-heuristic?
You can use one of your suggested methods to calculate the relative improvement, but you need also to define a threshold value $\epsilon$ that determines when a relative improvement is negligible, and ...
- 37.1k
2
votes
Is there any domain in machine learning that solves a problem by using only analytical algorithms?
In some cases, you can solve a linear regression problem with an analytical (or closed-form) solution/expression (although this may not always be the best approach). See this answer for more details.
...
- 37.1k
1
vote
How is catastrophic cancellation dealt with in loss functions?
Catastrophic cancellation occurs when a function to optimise includes the difference between two estimates to close numbers. As those estimates approach their true values, the ratio of the estimated ...
- 26.6k
1
vote
Is there any domain in machine learning that solves a problem by using only analytical algorithms?
Honourable mention: Memory-based approaches
Although not analytic, memory-based models, such as k-nearest neighbours (k-NN) are very lightweight when learning, but have a higher cost to use the stored ...
- 26.6k
Only top scored, non community-wiki answers of a minimum length are eligible