1
$\begingroup$

I have been working on coding a CNN in python from scratch using numpy as a semester project and I think I have successfully implemented it up to backpropagation in the MaxPool Layers. However, my model seems to never converge whenever there is a Convolutional Layer(s) added. I am assuming there is a problem with the way I have implemented the backpropagation.

Most examples that I have seen for this implementation either really simplify it by using a one-channel input and a single one-channel filter, or just dive straight into the Mathematics which doesn't only not help but also confuses me more.

Here is the way I have tried to implement both Forward and Backward Propagation for multichannel inputs and outputs based on my own understanding and things I read online.

Forward Prop: Forward Propagation in Convolutional Layer

Backward Prop for Filter Gradients: enter image description here

Backward Prop for Input Gradients: enter image description here

Kindly point out anything that's wrong here. I have been working on this part for the last 2 days but there has to be a problem because my model never seems to converge.

Thanks!

$\endgroup$
2
  • $\begingroup$ I'm new to this, but what do you mean by not converging ? the error rates goes bigger ? $\endgroup$ May 16 at 15:54
  • $\begingroup$ Thank you for the response @johnjerrico. The cost keeps fluctuating up and down, never drops as you would expect. Tried changing learning rates from 1 to 1e-8. No change $\endgroup$ May 16 at 17:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.