When it comes to CNNs, I don't understand 2 things in the training process:

  1. How do I pass the error back when there are pooling layers between the convolutional layers?

  2. And if I know how it's done, can I train all the layers just like layers in normal Feed Forward Neural Nets?


Yes. You can train end-to-end. The introduction of convolution kernels with associated pooling layers to the sequence of forward feed operations on the signals does not change the basic principles.

  • Gradient descent estimates the incremental change required to converge on an optimal behavior.
  • The corrective error must be distributed, which is most efficiently done by employing the derivative of the activation function sequentially to each set of parameters (whether convolution kernels or matrices that attenuates inputs to activation vectors) from the output back to the input.

Consider studying Backpropagation In Convolutional Neural Networks on Jefkine.com, which clarifies the application of those principles with convolution-pooling pairs.

There is another approach, borrowing from wisdom gained in the development of analog feedback in instrumentation. There are times when a sequence of operations can be better trained with more than one feedback loop, which requires some determination of error or wellness at intermediate stages and breaks the system into segments that each train based on those intermediate criteria.

This other approach is hierarchical, and an overall convergence may be controlled by a higher level back propagation, considering each segment as a black box. As in analog circuitry, multiple degrees of freedom with semi-independent convergence mechanisms has been shown to allow for deeper sequences without a major loss of convergence reliability or accuracy.

  • $\begingroup$ Thanks, but just when i thought i got it, I again became a little confused: Is one filter of a layer applied to all feature maps of the layer before the same way and added up? $\endgroup$
    – Evator
    Sep 30 '18 at 15:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.