I have been working on coding a CNN in python from scratch using numpy as a semester project and I think I have successfully implemented it up to backpropagation in the MaxPool Layers. However, my model seems to never converge whenever there is a Convolutional Layer(s) added. I am assuming there is a problem with the way I have implemented the backpropagation.
Most examples that I have seen for this implementation either really simplify it by using a one-channel input and a single one-channel filter, or just dive straight into the Mathematics which doesn't only not help but also confuses me more.
Here is the way I have tried to implement both Forward and Backward Propagation for multichannel inputs and outputs based on my own understanding and things I read online.
Kindly point out anything that's wrong here. I have been working on this part for the last 2 days but there has to be a problem because my model never seems to converge.