I have a very simple question about Conv nets. I understand the whole principle, but only one thing is not well explained on the Internet.
If I have a 16 channels image that goes on a convolutional layer, and the trainable filters are 3 7x7 filters, meaning that its output has 3 channels, how does the conv layer do to go from 16 to 3 channels? What mathematical operation is applied?
Thanks for any clarification on this.
The reason why you go from 16 to 3 channels is that by convention filters span the entire depth of the input. Therefore, your filters would actually be 7x7x16 in order to cover all channels of the input.
The output of the convolution automatically has a depth equal to the number of filters (so in your case this is 3) because you have an (m x k) filter matrix, where m is the number of filters and k is the number of elements in the unrolled filter (in your case, m = 3 and k = 7 x 7 x 16 = 784, so the filter matrix is 3 x 784).
The input is usually unrolled according to the im2col procedure, where each tile corresponding to a single filter location is stretched into a column equal to the unrolled filter size. This is repeated for each filter location, so you end up with very large matrix of size (k x n), where k is the same as k above in the filter matrix, and n depends on your padding and stride.
Multiplying the (m x k) filter matrix with the (k x n) input matrix gives you an (m x n) output matrix, where m is the number of filters.
You can find some very nice visual explanations of the convolution procedure here and here.
Your input is having 16 channels of each of dimension m x n and there are 3 filter namely f1, f2 and f3 of dimensions fm x fn. We say that a filter is applied to a channel when it is superimposed on the image starting left-most, performing the operation of multiplying the weights of filter with the corresponding value in the image and then summing up to a single value and moving the filter to right (then down when it reaches rightmost part) across the image according to the stride of the filter.
As mentioned when a filter say f1 is applied to a channel say c, there is a single value. Now applying them to all channels we get 16 values and all of them are added up to a single value. f1 is moved according to the stride and the same operation is repeated to get an output with a single channel (number of rows and columns are determined by padding, stride, dilation and kernel size of the filers).
The aforesaid process is done by all the 3 filters giving rise to 3 channels. In this way the conv layer makes the input to go from 16 to 3 channels. More detailed explanations can be found here.
In short, its basically a magic of matrix multiplications. The dimensions of the weight matrix are such that w.r.t the input layer dimensions, the desired output layer dimensions are taken care of.
