# Tag Info

Accepted

### Is there a theoretical way to determine the best learning rate for gradient descent if the function is a simple known polynomial?

You are considering a one-dimensional parabola; in this case, it is easy. The derivative is twice the distance to the optimal point, so the optimal learning rate is always 0.5/a, where a is coef near ...
Accepted

### REINFORCE with Baseline update rule

The value $\delta$ is already representing a derivative equivalent to derivative of MSE loss for the difference between observed and predicted return. Multiplying it by the gradient of $\hat{v}$ to ...
• 31.5k
Accepted

### What do you mean by "updating based on a training example/batch" in Gradient Descent?

In batch gradient descent computing the loss function every time serves several purposes. While the value of the loss itself may not directly affect the backpropagation process, as you said monitoring ...
• 1,188
1 vote

### Can you explain the Hinton's comment "Rprop is equivalent to using the gradient, but also dividing by the size of the gradient"?

"Rprop is equivalent to using the gradient" means Rprop fundamentally relies on information about the sign of the gradient of each weight to determine the direction of weight updates. Like ...
• 1,188

Only top scored, non community-wiki answers of a minimum length are eligible