14
votes
Accepted
Is back-propagation applied for each data point or for a batch of data points?
Short answers
Is back-propagation applied immediately after getting the output for each input or after getting the output for all inputs in a batch?
You can perform back-propagation using (or after) ...
- 41.4k
13
votes
What exactly is averaged when doing batch gradient descent?
Introduction
First of all, it's completely normal that you are confused because nobody really explains this well and accurately enough. Here's my partial attempt to do that. So, this answer doesn't ...
- 41.4k
12
votes
Accepted
What is the relationship between gradient accumulation and batch size?
There isn't any explicit relation between the batch size and the gradient accumulation steps, except for the fact that gradient accumulation helps one to fit models with relatively larger batch sizes (...
- 271
11
votes
Accepted
Why is the learning rate generally beneath 1?
If the learning rate is greater than or equal to $1$ the Robbins-Monro condition
$$\sum _{{t=0}}^{{\infty }}a_{t}^{2}<\infty\label{1}\tag{1},$$
where $a_t$ is the learning rate at iteration $t$, ...
- 41.4k
11
votes
Accepted
Is neural networks training done one-by-one?
Should I be changing the weights/biases on every single sample before moving on to the next sample,
You can do this, it is called stochastic gradient descent (SGD) and typically you will shuffle the ...
- 33.3k
9
votes
Accepted
Should we also shuffle the test dataset when training with SGD?
Short answer
Shuffling affects learning (i.e. the updates of the parameters of the model), but, during testing or validation, you are not learning. So, it should not make any difference whether you ...
- 41.4k
7
votes
How do I choose the optimal batch size?
From the blog A Gentle Introduction to Mini-Batch Gradient Descent and How to Configure Batch Size (2017) by Jason Brownlee.
How to Configure Mini-Batch Gradient Descent
Mini-batch gradient descent ...
7
votes
How do I choose the optimal batch size?
Here are a few guidelines, inspired by the deep learning specialization course, to choose the size of the mini-batch:
If you have a small training set, use batch gradient descent (m < 200)
In ...
7
votes
How do I handle negative rewards in policy gradients with the cross-entropy loss function?
It depends on your loss function, but you probably need to tweak it.
If you are using an update rule like loss = -log(probabilities) * reward, then your loss is ...
- 171
4
votes
What exactly is averaged when doing batch gradient descent?
do I have to:
forward propagate
calculate error
calculate all gradients
...repeatedly over all samples in the batch, and then average all gradients and apply the weight change?
Yes, that is ...
- 33.3k
4
votes
Accepted
How are these equations of SGD with momentum equivalent?
The first two equations are equivalent. The last equation can be equivalent if you scale $\alpha$ appropriately.
Equation 1
Consider the equation from the Stanford slide:
$$
v_{t}=\rho v_{t-1}+\nabla ...
- 1,690
4
votes
How do I choose the optimal batch size?
The batch size can also have a significant impact on your model’s performance and the training time. In general, the optimal batch size will be lower than 32. In April 2018, Yann Lecun even tweeted:
...
- 41
4
votes
Why to use gradient accumulation?
This image from here nicely illustrates how gradient accumulation is performed:
Assuming infinite memory and compute we would be able to compute the gradient on the full batch, this would provide us ...
- 433
4
votes
Is there any way to train a neural network without using gradients?
Yes.
A prominent class of "gradient-free" algorithms in ML world is known as Evolution Strategies (ES). Evolutionary Algorithms, although existed for a long time, only a few have shown to ...
- 258
4
votes
Why is it called "batch" gradient descent if it consumes the full dataset before calculating the gradient?
You are correct, but requires final words:
In Batch GD, we take the average of all training data to update the parameters, hence, one step per epoch. That's very valid if you have a convex problem (i....
- 141
3
votes
Can stochastic gradient descent be properly used in any sample based learning algorithm in Reinforcement Learning?
First I will address the issue of Tabular methods. These do not use SGD at all. Although the updates are very similar to an SGD update there is no gradient here and so we are not using SGD. Many ...
- 5,030
3
votes
What is the difference between batch and mini-batch gradient decent?
It is really simple.
In gradient descent not using mini-batches, you feed your entire training set of data into the network and accumulate a cost function based on this full set of data. Then you ...
- 724
3
votes
How do I handle negative rewards in policy gradients with the cross-entropy loss function?
The cross-entropy loss will always be positive because the probability is in the range $[0, 1]$, so $-ln(p)$ will always be positive.
- 31
2
votes
Accepted
What's the rationale behind mini-batch gradient descent?
The basic idea behind mini-batch training is rooted in the exploration / exploitation tradeoff in local search and optimization algorithms.
You can view training of an ANN as a local search through ...
- 9,392
2
votes
Accepted
In the MINE paper, why is $\hat{G}_B$ biased, and how does the exponential moving average reduce the bias?
The lower bound in MINE is as follows:
$$\widehat{I(X;Z)}_n = \sup_{\theta\in\Theta} \mathbb{E}_{\mathbb{P}_{XZ}^{(n)}}[T_\theta] - \log{\mathbb{E}_{\mathbb{P}_X^{(n)} \otimes \hat{\mathbb{P}}_Z^{(n)}}...
2
votes
Accepted
Is it possible to use stochastic gradient descent at the beginning, then switch to batch gradient descent with only a few training examples?
There is a trade-off between the:
Memory capacity of computation device
Quality of gradient approximation
Generalization ability of the network
Memory capacity
I would say, that it is possible to ...
- 2,754
2
votes
Derivation of Sutton & Barto TD(λ) Weight Update Equation with Eligibility Traces
Your concern about the extra term seems due to a symbolic error. When you compute $\bf{z}_t$, it already incorporates the contribution of the previous eligibility trace $\bf{z}_{t-1}$ due to the term $...
- 5,507
1
vote
What does it mean if I trained my model with more steps per epoch than the total number of training images I have?
Welcome to AI.stackexchange! To answer your question more precisely, it would be helpful to provide a minimum working example of your code where we can see how you implemented your training loop.
...
- 92
1
vote
Accepted
Can MSE be used for NN categorical classification problems
There are at least two reasons, why cross-entropy loss is preferred over mean squared error in classification problems.
A theoretical reason
Both aforementioned losses are negative logarithmic ...
1
vote
How does SGD training error decrease in subsequent epochs when statistically, it requires that samples in subsequent epochs be i.i.d and they are not?
The samples might be the same in each epoch, but, before starting the new epoch, the training dataset might be randomly shuffled before splitting it into mini-batches (this is the default behaviour in ...
- 41.4k
1
vote
Accepted
Unclear fact about difference between Gradient Descent to Stochastic Gradient Decent in wikipedia
That paragraph is incomplete and unclear indeed.
Let's crack the difference with a concrete example: logistic regression.
The objective we want to minimize is:
$J_{train}(\theta)=\frac{1}{2m}\sum_{i=1}...
- 5,428
1
vote
Issue with graphical interpretation of the universal approximation theorem
The classical version of the universal approximation theorem states that, roughly, given a continuous function $f \colon [0, 1]^n \to [0, 1]^n$, there exists a single layer neural network and a set of ...
- 1,010
1
vote
Stochastic gradient descent does not behave as expected, even with different activation functions
Could you post the pseudocode of your backpropagation algorithm?
I recommend you start off as simple as possible (this includes your cost f(x), I would simply use Yexpected-Youtput) and see if it ...
- 212
1
vote
Is the choice of the optimiser relevant when doing object detection?
I have experimented with this to a small degree and have not noticed that much of an impact.
To date, Adam appears to give the best results on a variety of image data sets. I have found that "...
- 724
1
vote
Is neural networks training done one-by-one?
Ideally, you need to update weights by going over all the samples in the dataset. This is called as Batch Gradient Descent. But, as the no. of training examples increases, the computation becomes huge ...
- 174
Only top scored, non community-wiki answers of a minimum length are eligible