1
$\begingroup$

I was studying Convolutional Layers and some of their variations and I came across this post which says:

'For rgb vs greyscale, think about channels as feature maps for input layer and a filter gets applied on all feature map at once.'

I also watched some videos that say the same thing, however I can't quite understand how the process actually happens. Let's look at the image below:

enter image description here

Now, lets pretend that we feed an RGB image to a ConvLayer2D. Since we have an RGB image we also have 3 input channels, one for each color. Lets also suppose that the ConvLayer has 32 filters. From what I understood the output of this layer will be 32 feature maps which means that each filter will apply convolutions to each of the 3 input channels and combine its convolved feature into one feature map related to that same filter. But how does that mixing/combining actually work? Does it produce 3 convolved feature maps (one for each channel input) and then combine its elements by some adding and dividing each elements of said convolved features before putting them into the final feature map? Or does it achieve the 32 feature maps in some other way?

For example: ((Red-i00 + Green-i00 + Blue-i00) / 3 = FinalFeatureMapOfThatFilter-i00 ) Where i00 represents the top, leftmost position of the related convolved feature.

Thanks!

$\endgroup$

1 Answer 1

2
$\begingroup$

If you have a conv layer that has 3 input channels and 32 output channels (i.e. the number of filters), then you essentially have $3 \times 32$ convolution operations connecting every input channel to every output channel (unless you are doing depthwise-separable convolution which is a different story). This gives you $3 \times 32$ sets of convolution kernels.

More concretely, filter 0 contains 3 sets of convolution kernels which convolves respectively with the 3 input features, the 3 outputs of the convolution operations are then summed into output channel 0. And the same applies to filter 1, filter 2, ..., filter 31.

$\endgroup$
2
  • $\begingroup$ Thank you for your answer! Just to be clear, the summed element values of the convolution operations are not divided by the number of channel inputs prior to be put into the output channel related to said filter right? Thanks again! $\endgroup$
    – Blue Ross
    Commented Sep 6, 2022 at 14:55
  • $\begingroup$ @BlueRoss No they are not. Though during the weight initialization, the weights would be initialized smaller if there are more input channels, to prevent the features from exploding in the forward pass $\endgroup$
    – PeaBrane
    Commented Sep 6, 2022 at 18:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .