In all examples I've ever seen, the learning rate of an optimisation method is always less than $1$. However, I've never found an explanation as to why this is. In addition to that, there are some cases where having a learning rate bigger than 1 is beneficial, such as in the case of super-convergence.
Why is the learning rate generally less than 1? Specifically, when performing an update on a parameter, why is the gradient generally multiplied by a factor less than 1 (absolutely)?
If the learning rate is greater than or equal to $1$ the Robbins-Monro condition $$\sum _{{t=0}}^{{\infty }}a_{t}^{2}<\infty\label{1}\tag{1},$$
where $a_t$ is the learning rate at iteration $t$, does not hold (given that a number bigger than $1$ squared becomes a bigger number), so stochastic gradient descent is not generally guaranteed to converge to a minimum [1] (although the condition \ref{1} is a sum from $t=0$ to $t=\infty$, but, of course, we only iterate for a finite number of iterations). Moreover, note that, if the learning rate is bigger than $1$, you are essentially giving more weight to the gradient of the loss function than to the current value of the parameters (you give weight $1$ to the parameters).
This is probably the main reason why the learning rate is usually in the range $(0, 1)$ and there are methods to decay the learning rate, which can be beneficial (and there are several explanations of why this is the case [2]).
