1
$\begingroup$

I am asking this question for a better understanding of the concept of channels in images.

I am aware that a convolutional layer generates feature maps from a given image. We can adjust the size of the output feature map by proper padding and regulating strides.

But I am not sure whether there exist kernels for a single convolution layer that are capable of changing an {RGBA, RGB, Grayscale, binary} image into (any) another {RGBA, RGB, Grayscale, binary} image?

For example, I have a binary image of a cat, is it capable to convert it into an RGBA image of a cat? If no, can it at least convert a binary cat image into an RGBA image?

I am asking only from a theoretical perspective.

Improve this question
$\endgroup$
8
  • $\begingroup$ You say "I am not sure whether there exist kernels for a single convolution layer", but note that the kernels in a CNN are usually learned. Note also that you don't need CNNs, for example, to convert RGB images into grayscale/binary ones. There are algorithms to convert RGB images into grayscale images. I'm not fully sure how this question that you're asking is related to your confusion about channels. $\endgroup$
    – nbro
    Jul 31, 2021 at 12:55
  • $\begingroup$ I'm also not sure why you're so confused about this concept, as it's not really anything special (usually it just refers to the 3rd dimension of the image or feature map, i.e. would be a synonym for depth, although in the case of the images the depth has some meaning to us, as each slice, for example, in RGB images, corresponds to the values of the red, green and blue color, hence the name RGB) $\endgroup$
    – nbro
    Jul 31, 2021 at 12:56
  • $\begingroup$ Please check here. It has two arguments in_channels, out_channels. What is the purpose of them? Example shows 16 input channels and 33 output channels. I am aware about images with 1 channel, 3 channels and 4 channels. @nbro $\endgroup$
    – hanugm
    Jul 31, 2021 at 12:58
  • 1
    $\begingroup$ In the input layer of the CNN, typically, you will have in_channels == 1 or in_channels == 3. However, in hidden layers of CNNs, you can have in_channels == K for $K > 1$, because this corresponds to the depth of the feature map that you produced in the previous convolutional layer, which corresponds to the number of kernels that you applied to the input of the previous layer (I'm assuming a 2d convolution). $\endgroup$
    – nbro
    Jul 31, 2021 at 13:03
  • 1
    $\begingroup$ Yes, I think it's just an example. $\endgroup$
    – nbro
    Jul 31, 2021 at 13:10

1 Answer 1

Reset to default
0
$\begingroup$

No, because each output from a convolution layer only looks at a local region of the image. A convolution layer cannot do any global transformation, only local ones. Convolution layers must have translation invariance which means if it converts an eyeball to a tail at one position, it'll also convert the same eyeball to the same tail if it's found at a different position. If it's not overfitted, it will also convert similar eyeballs to similar tails. If you want only some eyeballs to become tails, you can't do that without introducing overfitting, or expanding the convolution size until the layer can see enough context to distinguish which eyeballs should become tails and which ones shouldn't.

If you want to change one image into a specific other image, and don't care what happens to all other images, it might be possible to create a convolution layer that does this transformation. The input image has to be different wherever the output image is different, or else the convolution layer won't be able to produce that difference in the output image. You would be teaching it to recognize the specific pixel patterns in the input image and generate the specific pixels in the output image. This would be an extreme case of overfitting and wouldn't work for any other input images.

The number of channels in the input and output image is irrelevant, except that more channels means the network has more data to learn from, obviously.

Improve this answer
$\endgroup$
10
  • $\begingroup$ The number of channels in the input and output image is irrelevant, except that more channels means the network has more data to learn from, obviously. does it mean it is possible to change image with $k$ channels with an image of $n$ channels?where $k \ne n$? $\endgroup$
    – hanugm
    Jul 30, 2021 at 22:33
  • $\begingroup$ @hanugm Yes, you can do this in any image processing software. Of course, if you change to a better format, it will not magically improve the data, and if you go to a worse format, the quality of the image will decrease. For example, you can convert a colour image to greyscale. $\endgroup$
    – user253751
    Jul 31, 2021 at 0:02
  • $\begingroup$ If possible, please add in answer that an image of $k_1$ channels can be converted by a convolution layer in to an image of $k_2$ channels where $k_1 \ne k_2$. $\endgroup$
    – hanugm
    Jul 31, 2021 at 0:17
  • $\begingroup$ @hanugm You think there is something magical about channels for some reason. There is not. $\endgroup$
    – user253751
    Jul 31, 2021 at 8:28
  • $\begingroup$ True. Due to my misconception about the word channels I am facing complications. $\endgroup$
    – hanugm
    Jul 31, 2021 at 8:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .