I've gone through several descriptions of CNNs online and they all leave out a crucial part as if it were trivial.
A "volume" of neurons consists of several parallel layers ("feature maps"), each the result of convolving with a different kernel.
Between volumes there is usually a step where layers are pooled and subsampled.
The next volume has a different number of parallel layers.
How do the feature maps from one volume connect to the feature maps of the next volume? Is it one-to-many? Many-to-many? Do N kernels apply to each of M feature maps in the first volume, yielding N*M feature maps in the second volume? Are these N kernels the same for each feature map in the first volume, or do different kernels apply to each one?
Or is the number of maps in the second volume not necessarily a multiple of the number in the first volume? If so, do maps in the first volume get cross-synthesized somehow? Or maybe different numbers of maps in the second volume follow from each one in the first?
Or is it some other of umpteen trillion possibilities?