What is the point of using 1D and 2D convolutions with a kernel size of 1 and 1x1 respectively?

Question

I understand the gist of what convolutional neural networks do and what they are used for, but I still wrestle a bit with how they function on a conceptual level. For example, I get that filters with kernel size greater than 1 are used as feature detectors, and that number of filters is equal to the number of output channels for a convolutional layer, and the number of features being detected scales with the number of filters/channels.

However, recently, I've been encountering an increasing number of models that employ 1- or 2D convolutions with kernel sizes of 1 or 1x1, and I can't quite grasp why. It feels to me like they defeat the purpose of performing a convolution in the first place.

What is the advantage of using such layers? Are they not just equivalent to multiplying each channel by a trainable, scalar value?

Answer 1

Traditional CNNs used for image classification (and related tasks) are composed of 1 or more fully connected layers (FCs), after the convolutional and pooling layers, which take as input the features extracted from the convolutional and pooling layers, in order to perform classification or regression.

One problem with FCs in CNNs is that the number of parameters can be very big, with respect to the number of parameters in the convolutional layers.

There are tasks, such as image segmentation, where this big number of parameters is not really needed. An example of a neural network that does not make use of fully connected layers but only uses convolutions, downsampling (aka pooling), and upsampling operations is the U-net, which is used for image segmentation. A neural network that only uses convolutions is known as a fully convolutional network (FCN). Here I give a detailed description of FCNs and $1 \times 1$, which should also answer your question.

In any case, to answer your question more directly, $1 \times 1$ convolutions have been used for image segmentation tasks, i.e. dense classification tasks, i.e. tasks where you want to assign a label to each pixel (or a group of pixels), as opposed to sparse classification tasks such as image classification (where the goal is to assign 1 label to the whole image). Moreover, in comparison with FC layers, they have fewer parameters and, more importantly, the number of parameters in an FCN does not depend on the dimensions of the images (as in the case of traditional CNNs), which is a good thing (especially, when your images have high resolutions), but typically it depends on the number of kernels and instances (of objects), in the case of instance segmentation.

The FCN paper discusses this reduction of the number of parameters (and computation time), so you should probably read this paper for more details.

Answer 2

Typically 1x1 convolutions are used for changing the number of channels. Each output channel is a linear combination of the input channels.

For example, if you perform a 1x1 convolution with only one output channel on an RGB image, then you get a grayscale image, whose intensity is a linear combination of the red, green, and blue values of the corresponding pixel (plus bias).

If you perform a 1x1 convolution with more than one output channel, then each channel is formed in the same way, as a linear combination of the input channels. You can think of it as multiple convolutions, whose output is stacked on top of each other. All these filters have different parameters.

Notice that if the output was equivalent to multiplying each channel by a scalar value, then you would always have the same number of inputs and outputs.