3
$\begingroup$

In a CNN, does each new filter have different weights for each input channel, or are the same weights of each filter used across input channels?

This question helps me a lot.

Let, I have RGB input image. (3 channels) Then each filter has n×n weights for one channel. It means, actually the filter has totally 3×n×n weights.

For channel R, it has own n×n filter.

For channel G, it has own n×n filter.

For channel B, it has own n×n filter.

After inner product, add them all to make one feature map. Am I right?

And then, my question starts here. For some purpose, I will only use greyscale images as input. So the input images always have the same values for each RGB channel.

Then, can I reduce the number of weights in the filters? Because in this case, using three different n×n filters and adding them is same with using one n×n filter that is the summation of three filters.

Does this logic hold on a trained network? I have a trained network for RGB image input, but it is too heavy to run in real time. But I only use the greyscale images as input, so it seems I can make the network less heavy (theoretically, almost 1/3 of original).

I'm quite new in this field, so detailed explanations will be really appreciated. Thank you.

Improve this question
$\endgroup$
6
  • $\begingroup$ So you have a pre trained CNN on rgb images...and now you want to use it on greyscale images? $\endgroup$
    – user9947
    Sep 13, 2018 at 10:19
  • $\begingroup$ Yes, right. And it's impossible to retrain because I don't have training set... $\endgroup$
    – Jun
    Sep 13, 2018 at 10:30
  • $\begingroup$ Sum up : No you don't reduce your number of weights by 3, you reduce your first layer number of weights by 3 only ! $\endgroup$
    – Jérémy Blain
    Sep 13, 2018 at 10:30
  • $\begingroup$ @Jun Then maybe you should stay with the former CNN model. Or maybe try to reduce the number of weights, but it's not obvious that the network works better.... $\endgroup$
    – Jérémy Blain
    Sep 13, 2018 at 10:31
  • $\begingroup$ @JérémyBlain Probably will not work. $\endgroup$
    – user9947
    Sep 13, 2018 at 10:32

2 Answers 2

Reset to default
2
$\begingroup$

After inner product, add them all to make one feature map. Am I right?

yes, you are right.

Then, can I reduce the number of weights in the filters? Because in this case, using three different n×n filters and adding them is same with using one n×n filter that is the summation of three filters.

If you have transformed the image into greyscale then you no longer need 3 filters. You should retrain your model on greyscale images. In a greyscale image the value of each pixel is a single sample representing only an amount of light (the light intensity).

The network will run faster if that is the only architectural change you make, but keep in mind that by converting the image to greyscale you will lose information and probably some of the predictive power of your network.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for your comment. So greyscale image is the one-channel image, right? But I can't retrain it because I don't have training samples. What if I use RGB image with same values for each channel? It would be same with one channel grey scale image. $\endgroup$
    – Jun
    Sep 13, 2018 at 10:30
  • 1
    $\begingroup$ No. It will almost certainly work badly. Each of the three RGB channels contains information about the light intensity for each of those colours. You may get some results by selecting a channel and its filter (e.g using only the red channel and filter for each image) and upscaling the weights, but this will certainly badly reduce the accuracy of your network. Also, if you have many hidden layers the performance improvement will be very small since the amount of computation after the first CNN layer will be exactly the same. $\endgroup$
    – Mike Vella
    Sep 13, 2018 at 10:42
  • $\begingroup$ Colour is very important to classify images. Imagine fruit for example, which rely a lot of the color of the object. $\endgroup$
    – Jérémy Blain
    Sep 13, 2018 at 12:39
0
$\begingroup$

After inner product, add them all to make one feature map. Am I right?

Yes you are right. Now I will try to perform a transformation to preserve accuracy, cannot say about efficiency of the method.

Note: I have not worked on such type of problem, but knowing the maths behind CNN I will try to solve the problem theoretically.

First you have to know the RGB to greyscale conversion formulae which have bben used. Here are some common schemes.

So let us say each pixel had values $r, g, b$ and you converted it to $x_1*r + x_2*g +x_3*b$. So initially for simplicity let us say the we are talking about the corner pixel and convolution scheme $valid$, so the corner pixel with $RGB$ channels get multiplied with values $w_r, w_g, w_b$ during convolution and gets summed up.

But now you only have one pixel which is $x_1*r + x_2*g +x_3*b$. Now let us multiply this by $\frac {w_r}{x_1} + \frac {w_g}{x_2} + \frac {w_b}{x_3}$. This will result in: $(w_r*r + w_g*g +w_b*b) + (\frac {w_r*(x_2*g + x_3*b)}{x_1} +\frac {w_g*(x_1*r + x_3*b)}{x_2} + \frac {w_b*(x_2*g + x_1*r)}{x_3})$.

Now we have to try to remove the 2nd term of the equation. The paramters $w_r, w_g, w_b, x_1, x_2, x_3$ are predetermined already. So taking a single term $r$ from the second term of the equation gives $r*(\frac {w_g*x_1}{x_2} + \frac {w_b*x_1}{x_3})$. The second term has pre-determined value, and for the $r$ I think somehow using modern image analysis techniques you can find some approximate value of possible $r$. Do this for $g$ and $b$ as well and subtract it from the aforementioned equation and you will finally get $(w_r*r + w_g*g +w_b*b)$ which was the term obtained by convolution of filters with $RGB$ images.

I have done all this hypothetically, such image analysis techniques might not exist, but it is still worth a try. Probably better methods to reduce the second term exists in mathematical literature. I will leave it upto mathematicians to point in the right direction.

Improve this answer
$\endgroup$
3
  • $\begingroup$ and for the r I think somehow using modern image analysis techniques you can find some approximate value of possible r. What sort of 'modern image analysis techniques? Once the image is greyscale the RGB vector has been collapsed into a scalar value. I suppose you could use something like a colourising autoencoder to try and estimate the colours back but this seems way outside the scope of this problem. Additionally, even if the OP does this then as he is calculating (wr∗r+wg∗g+wb∗b) anyway the whole point of the problem (to improve inference performance) is lost anyway. $\endgroup$
    – Mike Vella
    Sep 13, 2018 at 12:23
  • $\begingroup$ @MikeVella i think you can confine the value of rgb to certain bounds based on the coefficients x1, x2, x3 and x1*r + x2*b + x3*g. And then you can even average it if you have some more info. $\endgroup$
    – user9947
    Sep 13, 2018 at 12:26
  • $\begingroup$ @MikeVella though i must say i am not sure as i mentioned in the answer....its upto mathematicians to confirm or deny my claim, but in all cases i think of the second term as noise which can be reduced somewhat since our main goal is to get the last bit of accuracy. $\endgroup$
    – user9947
    Sep 13, 2018 at 12:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .