11

Should I be changing the weights/biases on every single sample before moving on to the next sample, You can do this, it is called stochastic gradient descent (SGD) and typically you will shuffle the dataset before working through it each time. or should I first calculate the desired changes for the entire lot of 1,000 samples, and only then start ...


6

Introduction First of all, it's completely normal that you are confused because nobody really explains this well and accurately enough. Here's my partial attempt to do that. So, this answer doesn't completely answer the original question. In fact, I leave some unanswered questions at the end (that I will eventually answer). The gradient is a linear ...


6

Short answers Is back-propagation applied immediately after getting the output for each input or after getting the output for all inputs in a batch? You can perform back-propagation using (or after) only one training input (also known as data point, example, sample or observation) or multiple ones (a batch). However, the loss function to train the neural ...


6

It depends on your loss function, but you probably need to tweak it. If you are using an update rule like loss = -log(probabilities) * reward, then your loss is high when you unexpectedly got a large reward—the policy will update to make that action more likely to realize that gain. Conversely, if you get a negative reward with high probability, this will ...


5

From the blog A Gentle Introduction to Mini-Batch Gradient Descent and How to Configure Batch Size (2017) by Jason Brownlee. How to Configure Mini-Batch Gradient Descent Mini-batch gradient descent is the recommended variant of gradient descent for most applications, especially in deep learning. Mini-batch sizes, commonly called “batch sizes” for brevity, ...


5

Here are a few guidelines, inspired by the deep learning specialization course, to choose the size of the mini-batch: If you have a small training set, use batch gradient descent (m < 200) In practice: Batch mode: long iteration times Mini-batch mode: faster learning Stochastic mode: lose speed up from vectorization The typically mini-batch sizes ...


5

If the learning rate is greater than or equal to $1$ the Robbins-Monro condition $$\sum _{{t=0}}^{{\infty }}a_{t}^{2}<\infty\label{1}\tag{1},$$ where $a_t$ is the learning rate at iteration $t$, does not hold (given that a number bigger than $1$ squared becomes a bigger number), so stochastic gradient descent is not generally guaranteed to converge to a ...


5

There isn't any explicit relation between the batch size and the gradient accumulation steps, except for the fact that gradient accumulation helps one to fit models with relatively larger batch sizes (typically in single-GPU setups) by cleverly avoiding memory issues. The core idea of gradient accumulation is to perform multiple backward passes using the ...


4

Yes. A prominent class of "gradient-free" algorithms in ML world is known as Evolution Strategies (ES). Evolutionary Algorithms, although existed for a long time, only a few have shown to scale well. Recently, the research group OpenAI managed to train Deep RL models with a specific variant of ES (with careful engineering). You can read this paper. ...


3

You are correct, but requires final words: In Batch GD, we take the average of all training data to update the parameters, hence, one step per epoch. That's very valid if you have a convex problem (i.e. smooth error). On the other hand, in the Stochastic GD, we take one training sample to go one step towards the optimum, then repeat the latter for every ...


3

Short answer Shuffling affects learning (i.e. the updates of the parameters of the model), but, during testing or validation, you are not learning. So, it should not make any difference whether you shuffle or not the test or validation data (unless you are computing some metric that depends on the order of the samples), given that you will not be computing ...


3

The batch size can also have a significant impact on your model’s performance and the training time. In general, the optimal batch size will be lower than 32 (in April 2018, Yann Lecun even tweeted "Friends don’t let friends use mini-batches larger than 32“). A small batch size ensures that each training iteration is very fast, and although a large ...


3

do I have to: forward propagate calculate error calculate all gradients ...repeatedly over all samples in the batch, and then average all gradients and apply the weight change? Yes, that is correct. You can save a bit of memory by summing gradients as you go. Once you have calculated the gradients for one example for the weights of one layer, then you ...


3

First I will address the issue of Tabular methods. These do not use SGD at all. Although the updates are very similar to an SGD update there is no gradient here and so we are not using SGD. Many Tabular methods are proven to converge, for instance the paper by Chris Watkins titled "Q-Learning" introduces and proves that Q-learning converges. Also ...


3

The first two equations are equivalent. The last equation can be equivalent if you scale $\alpha$ appropriately. Equation 1 Consider the equation from the Stanford slide: $$ v_{t}=\rho v_{t-1}+\nabla f(x_{t-1}) \\ x_{t}=x_{t-1}-\alpha v_{t}, $$ Let's evaluate the first few $v_t$ so that we can arrive at a closed form solution: $v_0 = 0 \\ v_1 = \rho v_0 + ...


3

It is really simple. In gradient descent not using mini-batches, you feed your entire training set of data into the network and accumulate a cost function based on this full set of data. Then you use gradient descent to adjust the network weights to minimize the cost. Then you repeat this process until you get a satisfactory level of accuracy. For example, ...


2

The basic idea behind mini-batch training is rooted in the exploration / exploitation tradeoff in local search and optimization algorithms. You can view training of an ANN as a local search through the space of possible parameters. The most common search method is to move all the parameters in the direction that reduces error the most (gradient decent). ...


2

The cross-entropy loss will always be positive because the probability is in the range $[0, 1]$, so $-ln(p)$ will always be positive.


2

The lower bound in MINE is as follows: $$\widehat{I(X;Z)}_n = \sup_{\theta\in\Theta} \mathbb{E}_{\mathbb{P}_{XZ}^{(n)}}[T_\theta] - \log{\mathbb{E}_{\mathbb{P}_X^{(n)} \otimes \hat{\mathbb{P}}_Z^{(n)}}[e^{T_\theta}]}$$ Here $\mathbb{\hat{P}^{(n)}}$ denotes the empirical distribution that we get from n i.i.d samples of $\mathbb{P}.$ Note that in the above ...


2

There is a trade-off between the: Memory capacity of computation device Quality of gradient approximation Generalization ability of the network Memory capacity I would say, that it is possible to process the whole dataset at once only for small enough dataset and image resolution (or any other measure of the data sample size - text sequence length, number ...


1

The classical version of the universal approximation theorem states that, roughly, given a continuous function $f \colon [0, 1]^n \to [0, 1]^n$, there exists a single layer neural network and a set of weights and biases such that this network approximates the given function $f$ arbitrarily well. It doesn't say anything about how you obtain such weights: the ...


1

Could you post the pseudocode of your backpropagation algorithm? I recommend you start off as simple as possible (this includes your cost f(x), I would simply use Yexpected-Youtput) and see if it works and then continue adding things. If it's your first time with neural networks, I recommend you check this link out and you could also try practising the ...


1

I have experimented with this to a small degree and have not noticed that much of an impact. To date, Adam appears to give the best results on a variety of image data sets. I have found that "adjusting" the learning rate during training is an effective means of improving model performance and has more impact than the selection of the optimizer. ...


1

The model size (i.e., the number of parameters) is completely independent of the batch size you use during the training. Whatever the batch size is, your model processes each input within the batch independently. Concretely, batch size affects how many samples your model has to process before it makes an update. For example, when batch size is 1, it ...


1

Ideally, you need to update weights by going over all the samples in the dataset. This is called as Batch Gradient Descent. But, as the no. of training examples increases, the computation becomes huge and training will be very slow. With the advent of deep learning, training size is in millions and computation using all training examples is very impractical ...


Only top voted, non community-wiki answers of a minimum length are eligible