The 2015 article Cyclical Learning Rates for Training Neural Networks by Leslie N. Smith gives some good suggestions for finding an ideal range for the learning rate.
The paper's primary focus is the benefit of using a learning rate schedule that varies learning rate cyclically between some lower and upper bound, instead of trying to choose a single fixed learning rate value. For this to work, you still need to select good lower and upper bounds, and Smith suggests training the model for a few epochs while increasing the learning rate between a large range of values. At first, the learning rate will be too small to make any progress at all. As the learning rate increases, eventually, the loss will begin to decrease, but, at some point, the learning rate will get too large, and the loss will stop decreasing and even begin increasing. Your ideal range consists of the learning rate values where the loss was decreasing steeply. After finding your range, you can reset the weights and biases on your model and restart training using whatever learning rate schedule you plan to use for training.
Here is a concrete example from one of my experiments:
In this case, I start my learning rate search at 1e-09 and plan to end with a learning rate of 0.99 (although I am actually able to stop sooner than that). Your experiment may require different search bounds, but you could always start with that and adjust things as needed. At first, the loss plot is flat, and then it begins to decrease, but is too gradual. At the first red line, loss starts to decrease sharply, and once it reaches the second red line, the plot has begun to level off, so I can end my search. For this particular experiment, my ideal learning rate range had a minimum of 4.01e-4 and a maximum of 2.58e-2.
For more information, I suggest reading this Keras Learning Rate Finder post, which contains more information on how the process works and a tutorial for how to program it using Keras and Tensorflow.