Skip to main content
Share Your Experience: Take the 2024 Developer Survey
6 votes
Accepted

Why is second-order backpropagation useful?

Second-order optimization algorithms like Hessian optimization have more information on the curvature of the loss function, so converge much, much faster than first-order optimization algorithms like ...
user3667125's user avatar
  • 1,590
4 votes
Accepted

What is numerical stability?

You can find a definition for "numerical stability" in mathworld wolframe: Numerical stability refers to how a malformed input affects the execution of an algorithm. In a numerically stable ...
OmG's user avatar
  • 1,816
2 votes

What are most commons methods to measure improvement rate in a meta-heuristic?

You can use one of your suggested methods to calculate the relative improvement, but you need also to define a threshold value $\epsilon$ that determines when a relative improvement is negligible, and ...
nbro's user avatar
  • 40.8k
2 votes

Is there any domain in machine learning that solves a problem by using only analytical algorithms?

In some cases, you can solve a linear regression problem with an analytical (or closed-form) solution/expression (although this may not always be the best approach). See this answer for more details. ...
nbro's user avatar
  • 40.8k
1 vote
Accepted

How is catastrophic cancellation dealt with in loss functions?

Catastrophic cancellation occurs when a function to optimise includes the difference between two estimates to close numbers. As those estimates approach their true values, the ratio of the estimated ...
Neil Slater's user avatar
  • 32.5k
1 vote

Is there any domain in machine learning that solves a problem by using only analytical algorithms?

Honourable mention: Memory-based approaches Although not analytic, memory-based models, such as k-nearest neighbours (k-NN) are very lightweight when learning, but have a higher cost to use the stored ...
Neil Slater's user avatar
  • 32.5k

Only top scored, non community-wiki answers of a minimum length are eligible