Let us imagine $x$ as a tensor containing 1000 RGB images, each of size $64 \times 32$.

>>> x = torch.randn(1000, 3, 64, 32)
>>> print(x.shape)
torch.Size([1000, 3, 64, 32])

I am using a 2d convolutional layer that converts RGB images to single channel (say grayscale) images

>>> in_ch = 3
>>> out_ch = 1
>>> m = nn.Conv2d(in_ch, out_ch, 3, 1, 1)
>>> print(m)
Conv2d(3, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))

I passed the tensor $x$ in the convolutional layer and obtained another tensor of 1000 grayscale images, each of size $64 \times 32$.

>>> output = m(x)
>>> print(output.shape)
torch.Size([1000, 1, 64, 32])

Now, I can say that my convolutional layer converted an RGB image into a grayscale image using 2d kernel.

How it is doing?

RGB image has 3 planes each of size $64 \times 32$. If a kernel of 2 dimensions is used, then we will get 3 planes in output, corresponding to R, G, and B. How is it possible to convert an image with 3 channels into an image with one channel using 2d kernel?

I can visualize easily if I use a 3d kernel since the kernel considers three channels simultaneously and produces a single feature map for an RGB image.


Yes, the kernel is 3D in this case - or 4D as in 3x3x3x1. In the general case you can have multiple output channels, making it 3x3x3x8 for example. The number of channels isn't a convolution dimension because the filter does not "slide"/"translate"/"move" over this dimension. It's still a 2D convolution, and then the channel part of this operation is thought of separately from the convolution part. The 4D kernel is a bunch of 2D kernels. If you have 1 input channel and 1 output channel then it's just one 2D kernel. Or you can think of it as a bunch of 3D kernels if you like. Or a single 4D kernel. These are just arrays of numbers... you can slice an array up however you like, if you can find a way to think about it.

Note the groups parameter of Conv2d, which affects how the channels are convolved. The default is 1, which means:

At groups=1, all inputs are convolved to all outputs.

If you set it to 3 (and 3 output channels) then the Conv2d layer would maintain the channel separation.

  • $\begingroup$ Okay! seems that kernel size is misleading. Actually the kernel size is not $3 \times 3$. It is $3 \times 3 \times 3$. $\endgroup$
    – hanugm
    Aug 2 '21 at 22:32
  • $\begingroup$ (+1) for Yes, it is really a 3D convolution (in this case), because I need to understand remaining to appreciate the answer further. $\endgroup$
    – hanugm
    Aug 2 '21 at 23:30
  • $\begingroup$ Yes, it is really a 3D kernel (in this case),? $\endgroup$
    – hanugm
    Oct 1 '21 at 7:26
  • 1
    $\begingroup$ @hanugm The convolution kernel is actually 3x3x3. There could also be multiple output channels so you can say it's 3x3x3x1. It's still a 2D convolution because there are two dimensions where the kernel translates/slides/moves. $\endgroup$
    – user253751
    Oct 1 '21 at 8:01
  • $\begingroup$ Ha. So, if possible try to change 3D to 2D in your answer. It may be misleading. $\endgroup$
    – hanugm
    Oct 1 '21 at 10:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.