Questions tagged [optimizers]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
1answer
21 views

What do we mean by “infrequent features”?

I am reading this blog post: https://ruder.io/optimizing-gradient-descent/index.html. In the section about AdaGrad, it says: It adapts the learning rate to the parameters, performing smaller updates (...
1
vote
3answers
1k views

What kind of optimizer is suggested to use for binary classification of similar images?

I have spent some time searching Google and wasn't able to find out what kind of optimization algorithm is best for binary classification when images are similar to one another. I'd like to read ...
6
votes
1answer
625 views

What is the formula for the momentum and Adam optimisers?

In the gradient descent algorithm, the formula to update the weight $w$, which has $g$ as the partial gradient of the loss function with respect to it, is: $$w\ -= r \times g$$ where $r$ is the ...