7
$\begingroup$

I've gone through several descriptions of CNNs online and they all leave out a crucial part as if it were trivial.

A "volume" of neurons consists of several parallel layers ("feature maps"), each the result of convolving with a different kernel.

Between volumes, there is usually a step where layers are pooled and subsampled.

The next volume has a different number of parallel layers.

How do the feature maps from one volume connect to the feature maps of the next volume? Is it one-to-many? Many-to-many? Do N kernels apply to each of the M feature maps in the first volume, yielding N*M feature maps in the second volume? Are these N kernels the same for each feature map in the first volume, or do different kernels apply to each one?

Or, is the number of maps in the second volume not necessarily a multiple of the number in the first volume? If so, do maps in the first volume get cross-synthesized somehow? Or, maybe different numbers of maps in the second volume follow from each one in the first?

Or, is it some other of umpteen trillion possibilities?

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
2
$\begingroup$

Short answer: One to many
Long answer:

The point is that you use a 3D convolution in a CNN. Each kernel has the size of n*m*C (C is the number of feature maps) and every feature map has its own kernel(=weights) and bias.

An example:

The size of layer 2 is 9x9x10 (stride 1, no padding), the kernel size is 3x3x10.

The dimension of the next layer would be 3x3xn (n are the number of kernels that layer 2 has).

Here I found a very clear explanation.

I hope this helps.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ How can there be no stride? It seems stride needs to be at least 1. Also, if the layer is 9x9x10 and the kernel size is 3x3x10, how can such a convolution result in multiple feature maps? Isn't the depth of the next layer 10-D+1, with kernel depth D? It seems like the setup you describe yields a depth of 1 for the next layer. $\endgroup$
    – MackTuesday
    Aug 24, 2017 at 20:28
  • $\begingroup$ Yes you are right. I meant stride 1. I edited my answer $\endgroup$
    – james
    Aug 25, 2017 at 6:59
  • $\begingroup$ the depth of the feature map n+1 is equal to the numbers of kernels you used in the feature map n. so if you just use 1 kernel the depth of the next feature map is 1. - each feature map has its own kernel - this is one of the main advantages of the CNN over the full connected NN: there are much less parameters to fit. $\endgroup$
    – james
    Aug 25, 2017 at 7:07
1
$\begingroup$

If you really want to understand what goes on in a CNN and visualize it, then you can give a read to this.

This course needs some prerequisites like having a certain level of idea of the forward and backward propagation and knowing how the network learns on itself.

CNN is just a way of moving from fully connected layers (every input connects to every output) to partially connected layers.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .