1
$\begingroup$

I search this kind of question for a while and I find many discussions involve on counting the number of parameters of a Convolutional Neural Network, but not on the inputs. Using the Fashion MNIST dataset as an example, each black and white image has $28 \times 28 \times 1$ pixels and there are 60,000 images in the training dataset. Does that mean we have total number of $28 \times 28 \times 60,000=47,040,000$ inputs for the input layer of CNN?

My partner critics my baseline/simplest CNN model (for demo purpose) with just one convolutional layer with 10 filters/kernels (the kernel size is $3 \times 3$ ), padding has been used and strides = 1. The Keras model information is listed below. He says the training set only have a sample size of 60,000, but you have 78,510 parameters. He concerns over-fitting issues because I have more parameters than the inputs.

enter image description here

I really don't know how to explain the concepts to him clearly that the inputs of CNN are pixels. Could anyone help? A more detailed explanation will be very helpful and I am also happy to learn!

$\endgroup$

1 Answer 1

1
$\begingroup$

Let's say you are training a CNN to classify whether an image has a dog or a cat. Your dataset has only one (grayscale) photo of each, but they have a high resolution of 6000 x 4000 pixels. Wouldn't you expect the network to overfit, even though you have 2 x 6000 x 4000 = 48 million input pixels?

It will overfit, because after all it has only seen one dog and one cat. It might learn to make the distinction based on the background, or other irrelevant information. The network needs sufficiently many examples of each class to learn what is relevant and what isn't. Increasing the number of pixels / image doesn't change this.

$\endgroup$
1
  • $\begingroup$ Thank you. Are there any rules apply to CNN to compare the number of inputs and the number of parameters? And also for the sample size, for sample size, I am referring to the data points here. For traditional models, we use rule of thumb. $\endgroup$ Jul 1, 2022 at 12:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .