# Tag Info

Accepted

### What is numerical stability?

You can find a definition for "numerical stability" in mathworld wolframe: Numerical stability refers to how a malformed input affects the execution of an algorithm. In a numerically stable ...
• 1,723
Accepted

### Why is second-order backpropagation useful?

Second-order optimization algorithms like Hessian optimization have more information on the curvature of the loss function, so converge much, much faster than first-order optimization algorithms like ...
• 1,400

### What are most commons methods to measure improvement rate in a meta-heuristic?

You can use one of your suggested methods to calculate the relative improvement, but you need also to define a threshold value $\epsilon$ that determines when a relative improvement is negligible, and ...
• 37.1k

### Is there any domain in machine learning that solves a problem by using only analytical algorithms?

In some cases, you can solve a linear regression problem with an analytical (or closed-form) solution/expression (although this may not always be the best approach). See this answer for more details. ...
• 37.1k
1 vote

### How is catastrophic cancellation dealt with in loss functions?

Catastrophic cancellation occurs when a function to optimise includes the difference between two estimates to close numbers. As those estimates approach their true values, the ratio of the estimated ...
• 26.6k
1 vote

### Is there any domain in machine learning that solves a problem by using only analytical algorithms?

Honourable mention: Memory-based approaches Although not analytic, memory-based models, such as k-nearest neighbours (k-NN) are very lightweight when learning, but have a higher cost to use the stored ...
• 26.6k

Only top scored, non community-wiki answers of a minimum length are eligible