I once read somewhere that there is a range of learning rate within which learning is optimal in almost all the cases, but I can't find any literature about it. All I could get is the following graph from the paper: The need for small learning rates on large problems
IsIn the context of neural networks trained with gradient descent, is there a range of the learning rate, which should be used to savereduce the time of training time and get a good performance in almost all problems?