Let us imagine $x$ as a tensor containing 1000 RGB images, each of size $64 \times 32$.

>>> x = torch.randn(1000, 3, 64, 32)
>>> print(x.shape)
torch.Size([1000, 3, 64, 32])

I am using a 2d convolutional layer that converts RGB images to single channel (say grayscale) images

>>> in_ch = 3
>>> out_ch = 1
>>> m = nn.Conv2d(in_ch, out_ch, 3, 1, 1)
>>> print(m)
Conv2d(3, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))

I passed the tensor $x$ in the convolutional layer and obtained another tensor of 1000 grayscale images, each of size $64 \times 32$.

>>> output = m(x)
>>> print(output.shape)
torch.Size([1000, 1, 64, 32])

Now, I can say that my convolutional layer converted an RGB image into a grayscale image using 2d kernel.

How it is doing?

RGB image has 3 planes each of size $64 \times 32$. If a kernel of 2 dimensions is used, then we will get 3 planes in output, corresponding to R, G, and B. How is it possible to convert an image with 3 channels into an image with one channel using 2d kernel?

I can visualize easily if I use a 3d kernel since the kernel considers three channels simultaneously and produces a single feature map for an RGB image.


1 Answer 1


Yes, the kernel is 3D in this case - or 4D as in 3x3x3x1. In the general case you can have multiple output channels, making it 3x3x3x8 for example. The number of channels isn't a convolution dimension because the filter does not "slide"/"translate"/"move" over this dimension. It's still a 2D convolution, and then the channel part of this operation is thought of separately from the convolution part. The 4D kernel is a bunch of 2D kernels. If you have 1 input channel and 1 output channel then it's just one 2D kernel. Or you can think of it as a bunch of 3D kernels if you like. Or a single 4D kernel. These are just arrays of numbers... you can slice an array up however you like, if you can find a way to think about it.

Note the groups parameter of Conv2d, which affects how the channels are convolved. The default is 1, which means:

At groups=1, all inputs are convolved to all outputs.

If you set it to 3 (and 3 output channels) then the Conv2d layer would maintain the channel separation.

  • $\begingroup$ Okay! seems that kernel size is misleading. Actually the kernel size is not $3 \times 3$. It is $3 \times 3 \times 3$. $\endgroup$
    – hanugm
    Aug 2, 2021 at 22:32
  • $\begingroup$ (+1) for Yes, it is really a 3D convolution (in this case), because I need to understand remaining to appreciate the answer further. $\endgroup$
    – hanugm
    Aug 2, 2021 at 23:30
  • $\begingroup$ Yes, it is really a 3D kernel (in this case),? $\endgroup$
    – hanugm
    Oct 1, 2021 at 7:26
  • 1
    $\begingroup$ @hanugm The convolution kernel is actually 3x3x3. There could also be multiple output channels so you can say it's 3x3x3x1. It's still a 2D convolution because there are two dimensions where the kernel translates/slides/moves. $\endgroup$
    – user253751
    Oct 1, 2021 at 8:01
  • $\begingroup$ Ha. So, if possible try to change 3D to 2D in your answer. It may be misleading. $\endgroup$
    – hanugm
    Oct 1, 2021 at 10:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .