I'm suffering from a significant brain fart while trying to get my head around how does batch size affect overall model size e.g for CNNs. Does it serve as an additional dimension for all the weight tensors?
Considering:
- VGG16 model
- batch_size of 16
- image size of 224x224x3
- conv_1 being the initial 1x1 convolution with stride 1 and 3:64 channels mapping
The input will be a tensor of [16, 224, 224, 3]
shape. Will the output of convolution layer be [16, 224, 224, 64]
and therefore - will all the weights have additional 'batch size' dimension and thus - impose a linear increase of model size with respect to the batch size?