Questions tagged [batch-size]
For questions related to the notion of batch size in machine learning.
14
questions
0
votes
1
answer
37
views
Why should data batches in a neural network have an equal size?
Why should data batches in a neural network have an equal size?
I have seen some recent research works on making the batch size dynamic, but still, I can't find an answer to my question.
- 123
0
votes
1
answer
20
views
What can be an example other than batch normalization that uses statistics of batches?
Consider the following paragraph, taken from OPTIMIZING BATCHES of the textbook named Deep Learning with PyTorch by Eli Stevens et al., regarding the reasons for processing data into batches
The ...
- 3,099
0
votes
2
answers
91
views
Is it true that batch size of form $2^k$ gives better results?
I am confused among the following in selecting the batch size for my model.
#1: powers of 2
I generally see that batch sizes are in powers of two: 32, 64, 128, 256.
#2: maximum GPU
Suppose my GPU ...
- 3,099
0
votes
2
answers
117
views
Is there any relationship between the batch size and the number of epochs?
I am currently running a program with a batch size of 17 instead of batch size 32. The benchmark results are obtained at a batch size of 32 with the number of epochs 700.
Now I am running with batch ...
- 3,099
0
votes
1
answer
40
views
Will there be any changes in the model's performance due to the usage of very small batch sizes?
I am trying to run a code that has a batch size around 28. I can run the program on my GPU with this batch size.
But, when I modify the code for my requirements and try to run, it is showing an run-...
- 3,099
3
votes
0
answers
707
views
Why would a VAE train much better with batch sizes closer to 1 over batch size of 100+?
I've been training a VAE to reconstruct human names and when I train it on a batch size of 100+ after about 5 hours of training it tends to just output the same thing regardless of the input and I'm ...
- 597
1
vote
1
answer
353
views
Are there any rules for choosing batch size? [duplicate]
I am training a CNN with a batch size of 128, but I have some fluctuations in the validation loss, which are greater than one. I want to increase my batch size to 150 or 200, but, in the code examples ...
- 120
10
votes
1
answer
5k
views
What is the relationship between gradient accumulation and batch size?
I am currently training some models using gradient accumulation since the model batches do not fit in GPU memory. Since I am using gradient accumulation, I had to tweak the training configuration a ...
- 1,018
5
votes
2
answers
995
views
What is the purpose of the batch size in neural networks?
Why is a batch size needed to update the weights of a neural network?
According to that Youtube Video from 3B1B, the weights are updated by calculating the error between expectation and outcome of ...
- 435
8
votes
2
answers
3k
views
Effect of batch size and number of GPUs on model accuracy
I have a data set that was split using a fixed random seed and I am going to use 80% of the data for training and the rest for validation.
Here are my GPU and batch size configurations
use ...
- 241
2
votes
1
answer
233
views
What is the difference between validation percentage and batch size?
I'm doing transfer learning using Inception on Tensorflow. The code that I used for training is https://raw.githubusercontent.com/tensorflow/hub/master/examples/image_retraining/retrain.py
If you take ...
- 195
1
vote
1
answer
140
views
What is the reason behind using a test batch size?
If one examines the SSD: Single Shot MultiBox Detector code from this GitHub repository, it can be seen that, for a testing phase (evaluating network on test data set), there is a parameter ...
- 123
18
votes
3
answers
45k
views
How do I choose the optimal batch size?
Batch size is a term used in machine learning and refers to the number of training examples utilised in one iteration. The batch size
can be one of three options:
batch mode: where the batch size is ...
4
votes
2
answers
3k
views
Is there a way to translate the concept of batch size into reinforcement learning?
I am using a neural network as my function approximator for reinforcement learning. In order to get it to train well, I need to choose a good learning rate. Hand-picking one is difficult, so I read up ...
- 41