All Questions
Tagged with comparison optimization
11 questions
1
vote
1
answer
1k
views
When to use Pruning, Quantization , Distillation and others when optimizing speed
I want to understand how to optimize models for inference speed and am seeking some advice and best practices for the same.
I am a little bit aware of the concepts of pruning, quantization, and ...
- 789
6
votes
2
answers
5k
views
What are the advantages of the Kullback-Leibler over the MSE/RMSE?
I've recently encountered different articles that are recommending to use the KL divergence instead of the MSE/RMSE (as the loss function), when trying to learn a probability distribution, but none of ...
- 422
12
votes
1
answer
893
views
What are hyper-heuristics, and how are they different from meta-heuristics?
I wanted to know what the differences between hyper-heuristics and meta-heuristics are, and what their main applications are. Which problems are suited to be solved by hyper-heuristics?
- 41.4k
10
votes
1
answer
3k
views
What is the difference between reinforcement learning and evolutionary algorithms?
What is the difference between reinforcement learning (RL) and evolutionary algorithms (EA)?
I am trying to understand the basics of RL, but I do not yet have practical experience with RL. I know ...
- 41.4k
3
votes
1
answer
2k
views
How are these equations of SGD with momentum equivalent?
I know this question may be so silly, but I can not prove it.
In Stanford slide (page 17), they define the formula of SGD with momentum like this:
$$
v_{t}=\rho v_{t-1}+\nabla f(x_{t-1})
\\
x_{t}=x_{...
- 1,690
1
vote
1
answer
1k
views
What are the conceptual differences between regularisation and optimisation in deep neural nets?
I'm trying to wrap my mind around the concepts of regularisation and optimisation in neural nets, especially around their differences.
In my current understanding, regularisation is intended to tackle ...
- 41.4k
1
vote
0
answers
367
views
What is the difference between exploitation and exploration in the context of optimization?
In the paper Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigm (2015, published in Knowledge-Based Systems)
The test functions are divided to three groups: unimodal, multi-...
- 41.4k
0
votes
1
answer
87
views
What is the difference between the study of evolutionary algorithms and optimization?
I have a course named "Evolutionary Algorithms", but our teacher is always mentioning the word "optimization" in his lectures.
I am confused. Is he actually teaching optimization? If yes, why is the ...
- 41.4k
1
vote
1
answer
312
views
What is the difference between simulated annealing and deterministic annealing?
Not sure if this is the right place, but I was wondering if someone could briefly explain to me the differences & similarities between simulated annealing and deterministic annealing?
I know that ...
- 628
2
votes
1
answer
2k
views
What are advantages of using meta-heuristic algorithms on optimization problems?
What are the advantages and disadvantages of using meta-heuristic algorithms on optimization problems? Simply, why do we use meta-heuristic algorithms, like PSO, over traditional mathematical ...
- 41.4k
4
votes
1
answer
3k
views
What's the difference between RMSE and Euclidean distance, and when to use a custom loss? [closed]
I'm searching for a loss function that fits my project. Actually, I have two questions, but they are in the same direction. I take a look at the definition of the root mean squared error (RMSE) and ...
- 41.4k