# Confusion about conversion of RGB image to grayscale image using a convolutional layer with 2-dimensional filters

Let us imagine $$x$$ as a tensor containing 1000 RGB images, each of size $$64 \times 32$$.

>>> x = torch.randn(1000, 3, 64, 32)
>>> print(x.shape)
torch.Size([1000, 3, 64, 32])


I am using a 2d convolutional layer that converts RGB images to single channel (say grayscale) images

>>> in_ch = 3
>>> out_ch = 1
>>> m = nn.Conv2d(in_ch, out_ch, 3, 1, 1)
>>> print(m)
Conv2d(3, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))


I passed the tensor $$x$$ in the convolutional layer and obtained another tensor of 1000 grayscale images, each of size $$64 \times 32$$.

>>> output = m(x)
>>> print(output.shape)
torch.Size([1000, 1, 64, 32])


Now, I can say that my convolutional layer converted an RGB image into a grayscale image using 2d kernel.

How it is doing?

RGB image has 3 planes each of size $$64 \times 32$$. If a kernel of 2 dimensions is used, then we will get 3 planes in output, corresponding to R, G, and B. How is it possible to convert an image with 3 channels into an image with one channel using 2d kernel?

I can visualize easily if I use a 3d kernel since the kernel considers three channels simultaneously and produces a single feature map for an RGB image.

• Aug 2, 2021 at 22:32

Yes, the kernel is 3D in this case - or 4D as in 3x3x3x1. In the general case you can have multiple output channels, making it 3x3x3x8 for example. The number of channels isn't a convolution dimension because the filter does not "slide"/"translate"/"move" over this dimension. It's still a 2D convolution, and then the channel part of this operation is thought of separately from the convolution part. The 4D kernel is a bunch of 2D kernels. If you have 1 input channel and 1 output channel then it's just one 2D kernel. Or you can think of it as a bunch of 3D kernels if you like. Or a single 4D kernel. These are just arrays of numbers... you can slice an array up however you like, if you can find a way to think about it.

Note the groups parameter of Conv2d, which affects how the channels are convolved. The default is 1, which means:

At groups=1, all inputs are convolved to all outputs.

If you set it to 3 (and 3 output channels) then the Conv2d layer would maintain the channel separation.

• Okay! seems that kernel size is misleading. Actually the kernel size is not $3 \times 3$. It is $3 \times 3 \times 3$. Aug 2, 2021 at 22:32
• (+1) for Yes, it is really a 3D convolution (in this case), because I need to understand remaining to appreciate the answer further. Aug 2, 2021 at 23:30
• Yes, it is really a 3D kernel (in this case),? Oct 1, 2021 at 7:26
• @hanugm The convolution kernel is actually 3x3x3. There could also be multiple output channels so you can say it's 3x3x3x1. It's still a 2D convolution because there are two dimensions where the kernel translates/slides/moves. Oct 1, 2021 at 8:01
• Ha. So, if possible try to change 3D to 2D in your answer. It may be misleading. Oct 1, 2021 at 10:47