2
$\begingroup$

If I have a convolutional neuronal network, does the input dimension change the number of parameters? And if yes, why? If the sizes and lengths of the filters are still the same, how can the number of parameter in a network increase?

$\endgroup$
1
$\begingroup$

If I have a convolutional neuronal network, does the input dimension change the number of parameters? And if yes, why?

If the convolutional neural network (CNN) only uses convolutional layers, then the number of parameters does not increase as a function of the spatial dimensions ($x$ and $y$) of the input. This is one of the advantages of CNNs!

The reason is quite simple: the parameters of the convolutional layers are the kernels (aka filters), which typically have fixed dimensions and can, nevertheless, be applied to inputs of different spatial dimensions, provided that the necessary padding is used. However, note that padding can create bigger feature maps, but feature maps are not the parameters of the neural network: they are the output of the convolutional layers. That's probably what confuses you, when you see a diagram of a CNN, because you see bigger feature maps and you might think that the number of parameters increases.

Here's a TensorFlow 2 (and Keras) program that shows that the number of parameters does not change as a function of the $x$ and $y$ dimension of the input.

import tensorflow as tf

input_shapes = [(2 * k, 2 * k, 3) for k in range(2, 6)]

for input_shape in input_shapes:
    model = tf.keras.Sequential()
    model.add(tf.keras.layers.Input(shape=input_shape))
    model.add(tf.keras.layers.Conv2D(10, kernel_size=3, use_bias=True))
    model.summary() # The total number of parameters is always 280

The parameters of a convolutional layer can increase if you increase the size of each kernel and the number of kernels, but this does not necessarily depend on the input.

The parameters of the CNN can also increase if you increase the depth of the input, but that's typically fixed (either $3$ for RGB images or $1$ for grayscale images). The reason is quite simple too: the kernels in the first convolutional layer (connected to the input layer) will have the same depth as the depth of the input.

If your CNN also has fully connected layers, then the number of parameters also depends on the dimensions of the inputs. This is because the parameters of the fully connected layers depend on the number and dimensions of the feature maps (remember the flatten layer before the fully connected layers?), which, as I said, can increase as a function of the input.

If you don't want to use fully connected layers, you may want to try fully convolutional networks (FCNs), which do not make use of fully connected layers, but can, nonetheless, be used to solve classification (and other) tasks.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.