5
$\begingroup$

I understand the gist of what convolutional networks do and what they are used for, but I still wrestle a bit with how they function on a conceptual level. For example, I get that filters with kernel size >1 are used as feature detectors, and that number of filters = number of output channels for a convolutional layer and the number of features being detected scales with the number of filters/channels.

However, as of recently I've been encountering an increasing number of models that employ 1- or 2-D convolutions with kernel sizes of 1 or 1x1, and I can't quite grasp why. It feels to me like they defeat the purpose of performing a convolution in the first place. What is the advantage of using such layers? Are they not just equivalent to multiplying each channel by a trainable, scalar value?

$\endgroup$
  • 1
    $\begingroup$ Where have you been encountering those 1x1 convolutions? I'd guess that it's a matter of dimensionality reduction, but seeing the full architecture could suggest otherwise. This might help: stats.stackexchange.com/questions/194142/…. $\endgroup$ – Jean-François Savard Apr 28 '20 at 2:44
  • $\begingroup$ 1-D convolution is applied to time-series, 2-D to images and 3-D to volumetric data. A size one kernel doesn't make sense. $\endgroup$ – MightestDuck Sep 3 '20 at 10:45
  • $\begingroup$ Kernel size one is quite common, it allows you to increase/decrease the filter size. This question already has many good answers on cross validated: stats.stackexchange.com/questions/194142 $\endgroup$ – adamconkey Dec 13 '20 at 5:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.