11
votes
Accepted
Why is the learning rate generally beneath 1?
If the learning rate is greater than or equal to $1$ the Robbins-Monro condition
$$\sum _{{t=0}}^{{\infty }}a_{t}^{2}<\infty\label{1}\tag{1},$$
where $a_t$ is the learning rate at iteration $t$, ...
- 37k
6
votes
Is there an ideal range of learning rate which always gives a good result almost in all problems?
The 2015 article Cyclical Learning Rates for Training Neural Networks by Leslie N. Smith gives some good suggestions for finding an ideal range for the learning rate.
The paper's primary focus is the ...
- 161
6
votes
Accepted
Should I be decaying the learning rate and the exploration rate in the same manner?
First of all, I'd say that there is a reason to give Learning Rate (LR) and Exploration Rate (ER) the same decay: they play at the same scale (the number of successive batches you'll train your model ...
- 591
5
votes
Accepted
In Q-learning, shouldn't the learning rate change dynamically during the learning phase?
Yes you can decay the learning rate in Q-learning, and yes this should result in more accurate Q-values in the long term for many environments.
However, this is something that is harder to manage ...
- 26.5k
5
votes
What causes a model to require a low learning rate?
Gradient Descent is a method to find the optimum parameter of the hypothesis or minimize the cost function.
where alpha is learning rate
If the learning rate is high then it can overshoot the ...
- 338
5
votes
Accepted
If $\alpha$ decreases over time, why is Q-learning guaranteed to converge?
Why is this a convergence criterion?
It is because $R$ and $S'$ are stochastic. A large learning rate applied when these values have variance would not converge to mean, but would wander around ...
- 26.5k
4
votes
Is there an ideal range of learning rate which always gives a good result almost in all problems?
The visualisation can be found in The need for small learning rates on large problems. This paper by D. Randall Wilson and Tony R. Martinez from 2001 investigates the role of learning rates in ...
3
votes
Accepted
Why is the validation loss less than the training loss, and what can be said about the effect of the learning rate?
This is very difficult to tell with the information provided, but the phenomenon is something that I have encountered many times before. Sometimes this is not a bad thing, here are some possible ...
- 218
3
votes
Is there a way to translate the concept of batch size into reinforcement learning?
Potentially.
If you do offline reinforcement learning, you're basically learning to approximate a function by sampling input/output pairs, rather than episode-by-episode. Here, your batch size could ...
- 9,037
2
votes
Is there a way to translate the concept of batch size into reinforcement learning?
From my understanding of reinforcement learning, you will have an agent and an environment.
In each episode, the agent observes the state $s$, takes some action action $a$, then gets some reward $r$, ...
- 353
2
votes
Is stable learning preferable to jumps in accuracy/loss
There is an approach to machine learning, called Simulated Annealing, which varies the rate: starting from a large rate, it is slowly reduced over time. The general idea is that the initial larger ...
- 5,242
2
votes
How to prove that gradient descent doesn't necessarily find the global optimum?
You can find by yourself a counterexample that, in general, GD is not guaranteed to find the global optimum!
I first advise you to choose a simpler function (than the one you are showing), with 2-3 ...
- 37k
2
votes
Is learning rate the only reason for training loss oscillation after few epochs?
The loss graph indicates that the model converged to a local minimum, already after a few epochs, and the weights start to oscillate around it. The learning rate is surely responsible for it, but it's ...
- 4,753
2
votes
Accepted
How does the learning rate $\alpha$ vary in stationary and non-stationary environments?
So why is constant-$\alpha$ being used?
This is because control scenarios are inherently non-stationary with respect to value functions. Decaying alpha comes with a risk that improvements to the ...
- 26.5k
1
vote
Effects of hyperparameters in Q-learning
Discount factor in (tabular) RL including Q-learning generally acts as a regularization hyperparameter to trade-off optimality with sample efficiency especially for continuous tasks with infinite time ...
- 561
1
vote
Can the optimal learning rate differ for different architectures?
Yes, the optimal learning rate will differ for every change you make in the network. In fact finding the optimal learning rate is very computationally expensive, so you will normally only get a rough ...
- 1,316
1
vote
Is stable learning preferable to jumps in accuracy/loss
If you have an erratic loss landscape, it can lead to an unstable learning curve. Thus, it's always better to choose a simpler function which creates a simple landscape. Sometimes even due to uneven ...
- 728
1
vote
Autoencoder network for feature selection not converging
The trick was to normalize the input dataset values with the respective mean and standard deviation in each column. This reduced the loss drastically, and my network is training more efficiently now. ...
- 51
1
vote
Accepted
Is this learning rate schedule increasing the learning rate?
The higher (or smaller) the learning rate, the higher (or, respectively, smaller) the contribution of the gradient of the objective function, with respect to the parameters of the model, to the new ...
- 37k
1
vote
Accepted
Why would the learning rate curve go backwards?
I have not used fastai library but this also happens on tensorboard when you have more than one training being recorded on the same plot.
Looking at the picture, I think this is a very special type of ...
- 1,058
1
vote
Is it harmful to set the learning rate of training a model to be too high if there is some decay function for the learning rate?
Setting too high a learning rate will extend the time to get a good result.
In my opinion, it is better to set not too big a learning rate but to use learning with momentum. When the learning starts ...
- 65
1
vote
Is it a good idea to overfit on a small part of your data for faster model convergence?
The first thing which you first have to understand is that does your trained model is working efficiently with both the training and testing data. If yes then its not overfitting. There is only one ...
- 70
1
vote
Accepted
How to prove that gradient descent doesn't necessarily find the global optimum?
Well, GD terminates once the gradients are 0, right? Now, in a non-convex function, there could be some points, which do not belong to the global minima, and yet, have 0 gradients. For example, such ...
- 890
1
vote
Why can the learning rate make the loss increase in stochastic gradient descent?
This is the case as the loss doesn't have to monotonically decrease when it's updated in the negative direction.
For example:
Let $L(\theta) = \theta^2 $ and $\theta_0= 3$
Let the subscript n in $\...
- 359
Only top scored, non community-wiki answers of a minimum length are eligible