# How do I calculate the partial derivative with respect to $x$?

I am trying to implement CNN using python NumPy. I searched so much, but all I found was for one filter with one channel for convolution.

Suppose $$x$$ is an image with the shape: (N_Height, N_Width, N_Channel) = (5,5,3).

Let's say I have 16 filters with this shape: (F_Height, F_Width, N_Channel) = (3,3,3) , stride=1 and padding=0

Forward:

The output shape after convolution 2d will be

(
math.floor((N_Height - F_Height + 2*padding)/stride + 1 )),
math.floor((N_Width- F_Width + 2*padding)/stride + 1 )),
filter_count
)


So, the output of this layer will be an array with this shape: (Height, Width, Channel) = (3, 3, 16)

BackPropagation:

Suppose $$dL/dh$$ is the input for my layer in back-propagation with this shape: (3, 3, 16)

Now, I must find $$dL/dw$$ and $$dL/dx$$: $$dL/dw$$ to update my filters parameter and $$dL/dx$$ to pass it as input to the previous layer as the loss respect to the input $$x$$.

From this answer Error respect to filters weights I found how to calculate $$dL/dw$$.

The problem I have in the back-propagation is I don't know how to calculate $$dL/dx$$ having this shape: (5, 5, 3) and pass it to the previous layer.

I read lots of articles in Medium and other sites, but I don't get how to calculate it:

## 1 Answer

While this may not be the answer you were looking for, I hope this explanation will help you to understand applying backpropagation to a CNN. Fundamentally, convolutional layers are no different than dense layers, however there are restrictions. The key one is weight-sharing which allows a CNN to be much more efficient than a regular dense layer (as well as it being sparse due to locality). Imagine we are transforming a 4x4 image into a 2x2 image. Since we are inputting a 16-vector, and outputting a 4-vector, we need a weights matrix of 4x16:

This has 64 parameters. In a convolutional layer, we can accomplish this by convolving a 3x3 kernel over the image:

$$K= \begin{bmatrix} k_{1,1} & k_{1,2} & k_{1,3} \\ k_{2,1} & k_{2,2} & k_{2,3} \\ k_{3,1} & k_{3,2} & k_{3,3} \end{bmatrix}$$

This convolution is equivalent to multiplying by the weights matrix:

As you can see, this only requires 9 parameters and backpropagation can be applied to update these parameters.

• thanks but as you said it is not my question's answer. I finally find $dL/dX$ for each channel in my example 16 then I add all of them and I get my $dL/dX$ Commented May 27, 2020 at 8:15