Questions tagged [batch-size]

For questions related to the notion of batch size in machine learning.

Filter by
Sorted by
Tagged with
1
vote
2answers
81 views

Is it true that batch size of form $2^k$ gives better results?

I am confused among the following in selecting the batch size for my model. #1: powers of 2 I generally see that batch sizes are in powers of two: 32, 64, 128, 256. #2: maximum GPU Suppose my GPU ...
0
votes
2answers
61 views

Is there any relationship between the batch size and the number of epochs?

I am currently running a program with a batch size of 17 instead of batch size 32. The benchmark results are obtained at a batch size of 32 with the number of epochs 700. Now I am running with batch ...
0
votes
1answer
35 views

Will there be any changes in the model's performance due to the usage of very small batch sizes?

I am trying to run a code that has a batch size around 28. I can run the program on my GPU with this batch size. But, when I modify the code for my requirements and try to run, it is showing an run-...
0
votes
0answers
27 views

The Effect of Batch Size in CNN Image Classifier, Given Different Image Domain

I am currently training image classifiers on CelebA face attribute dataset (Binary classifiers), which will be used on images generated by StyleGAN (that is trained on FFHQ256 dataset). These ...
3
votes
0answers
482 views

Why would a VAE train much better with batch sizes closer to 1 over batch size of 100+?

I've been training a VAE to reconstruct human names and when I train it on a batch size of 100+ after about 5 hours of training it tends to just output the same thing regardless of the input and I'm ...
1
vote
1answer
288 views

Are there any rules for choosing batch size? [duplicate]

I am training a CNN with a batch size of 128, but I have some fluctuations in the validation loss, which are greater than one. I want to increase my batch size to 150 or 200, but, in the code examples ...
7
votes
1answer
4k views

What is the relationship between gradient accumulation and batch size?

I am currently training some models using gradient accumulation since the model batches do not fit in GPU memory. Since I am using gradient accumulation, I had to tweak the training configuration a ...
5
votes
2answers
942 views

What is the purpose of the batch size in neural networks?

Why is a batch size needed to update the weights of a neural network? According to that Youtube Video from 3B1B, the weights are updated by calculating the error between expectation and outcome of ...
8
votes
2answers
2k views

Effect of batch size and number of GPUs on model accuracy

I have a data set that was split using a fixed random seed and I am going to use 80% of the data for training and the rest for validation. Here are my GPU and batch size configurations use ...
2
votes
1answer
208 views

What is the difference between validation percentage and batch size?

I'm doing transfer learning using Inception on Tensorflow. The code that I used for training is https://raw.githubusercontent.com/tensorflow/hub/master/examples/image_retraining/retrain.py If you take ...
1
vote
1answer
43 views

What is the reason behind using a test batch size?

If one examines the SSD: Single Shot MultiBox Detector code from this GitHub repository, it can be seen that, for a testing phase (evaluating network on test data set), there is a parameter ...
15
votes
3answers
36k views

How do I choose the optimal batch size?

Batch size is a term used in machine learning and refers to the number of training examples utilised in one iteration. The batch size can be one of three options: batch mode: where the batch size is ...
4
votes
2answers
3k views

Is there a way to translate the concept of batch size into reinforcement learning?

I am using a neural network as my function approximator for reinforcement learning. In order to get it to train well, I need to choose a good learning rate. Hand-picking one is difficult, so I read up ...