9 votes
Accepted

Why is the learning rate generally beneath 1?

If the learning rate is greater than or equal to $1$ the Robbins-Monro condition $$\sum _{{t=0}}^{{\infty }}a_{t}^{2}<\infty\label{1}\tag{1},$$ where $a_t$ is the learning rate at iteration $t$, ...
user avatar
  • 34.4k
6 votes
Accepted

Should I be decaying the learning rate and the exploration rate in the same manner?

First of all, I'd say that there is a reason to give Learning Rate (LR) and Exploration Rate (ER) the same decay: they play at the same scale (the number of successive batches you'll train your model ...
user avatar
  • 571
5 votes

What causes a model to require a low learning rate?

Gradient Descent is a method to find the optimum parameter of the hypothesis or minimize the cost function. where alpha is learning rate If the learning rate is high then it can overshoot the ...
user avatar
  • 328
5 votes
Accepted

If $\alpha$ decreases over time, why is Q-learning guaranteed to converge?

Why is this a convergence criterion? It is because $R$ and $S'$ are stochastic. A large learning rate applied when these values have variance would not converge to mean, but would wander around ...
user avatar
  • 24.5k
5 votes

Is there an ideal range of learning rate which always gives a good result almost in all problems?

The 2015 article Cyclical Learning Rates for Training Neural Networks by Leslie N. Smith gives some good suggestions for finding an ideal range for the learning rate. The paper's primary focus is the ...
user avatar
4 votes

Is there an ideal range of learning rate which always gives a good result almost in all problems?

The visualisation can be found in The need for small learning rates on large problems. This paper by D. Randall Wilson and Tony R. Martinez from 2001 investigates the role of learning rates in ...
user avatar
4 votes
Accepted

In Q-learning, shouldn't the learning rate change dynamically during the learning phase?

Yes you can decay the learning rate in Q-learning, and yes this should result in more accurate Q-values in the long term for many environments. However, this is something that is harder to manage ...
user avatar
  • 24.5k
3 votes
Accepted

Why is the validation loss less than the training loss, and what can be said about the effect of the learning rate?

This is very difficult to tell with the information provided, but the phenomenon is something that I have encountered many times before. Sometimes this is not a bad thing, here are some possible ...
user avatar
  • 218
3 votes

Is there a way to translate the concept of batch size into reinforcement learning?

Potentially. If you do offline reinforcement learning, you're basically learning to approximate a function by sampling input/output pairs, rather than episode-by-episode. Here, your batch size could ...
user avatar
2 votes

Is there a way to translate the concept of batch size into reinforcement learning?

From my understanding of reinforcement learning, you will have an agent and an environment. In each episode, the agent observes the state $s$, takes some action action $a$, then gets some reward $r$, ...
user avatar
2 votes

How to prove that gradient descent doesn't necessarily find the global optimum?

You can find by yourself a counterexample that, in general, GD is not guaranteed to find the global optimum! I first advise you to choose a simpler function (than the one you are showing), with 2-3 ...
user avatar
  • 34.4k
2 votes

Is learning rate the only reason for training loss oscillation after few epochs?

The loss graph indicates that the model converged to a local minimum, already after a few epochs, and the weights start to oscillate around it. The learning rate is surely responsible for it, but it's ...
user avatar
2 votes

Is stable learning preferable to jumps in accuracy/loss

There is an approach to machine learning, called Simulated Annealing, which varies the rate: starting from a large rate, it is slowly reduced over time. The general idea is that the initial larger ...
user avatar
  • 5,062
2 votes
Accepted

How does the learning rate $\alpha$ vary in stationary and non-stationary environments?

So why is constant-$\alpha$ being used? This is because control scenarios are inherently non-stationary with respect to value functions. Decaying alpha comes with a risk that improvements to the ...
user avatar
  • 24.5k
1 vote

Can the optimal learning rate differ for different architectures?

Yes, the optimal learning rate will differ for every change you make in the network. In fact finding the optimal learning rate is very computationally expensive, so you will normally only get a rough ...
user avatar
  • 1,246
1 vote

Is stable learning preferable to jumps in accuracy/loss

If you have an erratic loss landscape, it can lead to an unstable learning curve. Thus, it's always better to choose a simpler function which creates a simple landscape. Sometimes even due to uneven ...
user avatar
1 vote
Accepted

Why would the learning rate curve go backwards?

I have not used fastai library but this also happens on tensorboard when you have more than one training being recorded on the same plot. Looking at the picture, I think this is a very special type of ...
user avatar
  • 1,038
1 vote

Is it harmful to set the learning rate of training a model to be too high if there is some decay function for the learning rate?

Setting too high a learning rate will extend the time to get a good result. In my opinion, it is better to set not too big a learning rate but to use learning with momentum. When the learning starts ...
user avatar
1 vote
Accepted

How to prove that gradient descent doesn't necessarily find the global optimum?

Well, GD terminates once the gradients are 0, right? Now, in a non-convex function, there could be some points, which do not belong to the global minima, and yet, have 0 gradients. For example, such ...
user avatar
1 vote

Autoencoder network for feature selection not converging

The trick was to normalize the input dataset values with the respective mean and standard deviation in each column. This reduced the loss drastically, and my network is training more efficiently now. ...
user avatar
1 vote
Accepted

Is this learning rate schedule increasing the learning rate?

The higher (or smaller) the learning rate, the higher (or, respectively, smaller) the contribution of the gradient of the objective function, with respect to the parameters of the model, to the new ...
user avatar
  • 34.4k
1 vote

Why can the learning rate make the loss increase in stochastic gradient descent?

This is the case as the loss doesn't have to monotonically decrease when it's updated in the negative direction. For example: Let $L(\theta) = \theta^2 $ and $\theta_0= 3$ Let the subscript n in $\...
user avatar
  • 349

Only top scored, non community-wiki answers of a minimum length are eligible