We changed our privacy policy. Read more.

Questions tagged [batch-gradient-descent]

For questions about mini-batch (or batch) gradient descent, which is gradient descent with typically more than one sample of input-label pairs.

Filter by
Sorted by
Tagged with
1
vote
1answer
54 views

Why is it called "batch" gradient descent if it consumes the full dataset before calculating the gradient?

While training a neural network, we can follow three methods: batch gradient descent, mini-batch gradient descent and stochastic gradient descent. For this question, assume that your dataset has $n$ ...
1
vote
1answer
47 views

Why does my model not improve when training with mini-batch gradient descent, while it does with Adam?

I am currently experimenting with the U-Net. I am doing semantic segmentation on the 2018 Data Science Bowl dataset from Kaggle without any data augmentation. In my experiments, I am trying different ...
5
votes
2answers
685 views

What exactly is averaged when doing batch gradient descent?

I have a question about how the averaging works when doing mini-batch gradient descent. I think I now understood the general gradient descent algorithm, but only for online learning. When doing mini-...
10
votes
2answers
904 views

Is neural networks training done one-by-one? [duplicate]

I'm trying to learn neural networks by watching this series of videos and implementing a simple neural network in Python. Here's one of the things I'm wondering about: I'm training the neural network ...
4
votes
1answer
2k views

Is back-propagation applied for each data point or for a batch of data points?

I am new to deep learning and trying to understand the concept of back-propagation. I have a doubt about when the back-propagation is applied. Assume that I have a training data set of 1000 images ...
1
vote
2answers
122 views

What's the rationale behind mini-batch gradient descent?

I am reading a book that states As the mini-batch size increases, the gradient computed is closer to the 'true' gradient So, I assume that they are saying that mini-batch training only focuses on ...
1
vote
1answer
433 views

What is the order of execution of steps in back-propagation algorithm in a neural network?

I am a machine learning newbie. I am trying to understand the back-propagation algorithm. I have a training dataset of 60 instances/records. What is the correct order of the process? This one? ...