How do I avoid my gradient descent algorithm into falling into the "local minima" trap while backpropogating on my neural network?
Are there any methods which help me avoid it?
Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this communityHow do I avoid my gradient descent algorithm into falling into the "local minima" trap while backpropogating on my neural network?
Are there any methods which help me avoid it?
There are several elementary techniques to try and move a search out of the basin of attraction of local optima. They include:
See the excellent (and free online) book 'Essentials of Metaheuristics' by Sean Luke for more details on these kind of techniques and some rules of thumb about when and how to use them.