Return to Question

Assume in a convolutional layer's forward pass we have a $10\times10\times3$ image and five $3\times3\times3$ kernels, then $(10\times10\times3) *( 3\times3\times3\times5)$ has the output of dimensions $8\times8\times5$. Therefore the gradients fed backwards to this convolutional layer also have the dimensions $8\times8\times5$.

When calculating the derivative of loss w.r.t. kernels, the formula is the convolution $input \times \frac{dL}{dZ}$. But if the gradients have dimensions $8\times8\times5$, how is it possible to convolve it with $10\times10\times3$? The gradients have $5$ channels while the input only has $3$.

Since during the forward pass the kernel window does element-wise multiplication and brings the channels down to $1$, do the gradients propagate back to each of the $3$ channels equally? Should the $8\times8\times5$ gradients be reshaped into $8\times8\times1\times5$ and broadcast into $8\times8\times3\times5$ before convolving with the layer input?

Assume in a convolutional layer's forward pass we have a $10\times10\times3$ image and five $3\times3\times3$ kernels, then $(10\times10\times3) *( 3\times3\times3\times5)$ has the output of dimensions $8\times8\times5$. Therefore the gradients fed backwards to this convolutional layer also have the dimensions $8\times8\times5$.

When calculating the derivative of loss w.r.t. kernels, the formula is the convolution $input * \frac{dL}{dZ}$. But if the gradients have dimensions $8\times8\times5$, how is it possible to convolve it with $10\times10\times3$? The gradients have $5$ channels while the input only has $3$.

Since during the forward pass the kernel window does element-wise multiplication and brings the channels down to $1$, do the gradients propagate back to each of the $3$ channels equally? Should the $8\times8\times5$ gradients be reshaped into $8\times8\times1\times5$ and broadcasted into $8\times8\times3\times5$ before convolving with the layer input?

Assume in a convolutional layer's forward pass we have a 10x10x3 image and 5 3x3x3 kernel, then 10x10x3 * 3x3x3x5 has the output of dimensions 8x8x5. Therefore the the gradients fed backwards to this convolutional layer also have the dimensions 8x8x5.

When calculating the derivative of loss w.r.t. kernels, the formula is the convolution $input * \frac{dL}{dZ}$. But if the gradients have dimensions 8x8x5, how is it possible to convolve it with 10x10x3? The gradients have 5 channels while the input only has 3.

Since during the forward pass the kernel window does element-wise multiplication and brings the channels down to 1, do the gradients propagate back to each of the 3 channel equally? Should the 8x8x5 gradients be reshaped into 8x8x1x5 and broadcasted into 8x8x3x5 before convolving with the layer input?

Assume in a convolutional layer's forward pass we have a $10\times10\times3$ image and five $3\times3\times3$ kernels, then $(10\times10\times3) *( 3\times3\times3\times5)$ has the output of dimensions $8\times8\times5$. Therefore the gradients fed backwards to this convolutional layer also have the dimensions $8\times8\times5$.

When calculating the derivative of loss w.r.t. kernels, the formula is the convolution $input \times \frac{dL}{dZ}$. But if the gradients have dimensions $8\times8\times5$, how is it possible to convolve it with $10\times10\times3$? The gradients have $5$ channels while the input only has $3$.

Since during the forward pass the kernel window does element-wise multiplication and brings the channels down to $1$, do the gradients propagate back to each of the $3$ channels equally? Should the $8\times8\times5$ gradients be reshaped into $8\times8\times1\times5$ and broadcast into $8\times8\times3\times5$ before convolving with the layer input?

Assume in a convolutional layer's forward pass we have a 10x10x3 image and 5 3x3x3 kernel, then 10x10x3 * 3x3x3x5 has the output of dimensions 8x8x5. Therefore the the gradients fed backwards to this convolutional layer also have the dimensions 8x8x5.

When calculating the derivative of loss w.r.t. kernels, the formula is the convolution $input * \frac{dL}{dZ}$. But if the gradients have dimensions 8x8x5, how is it possible to convolve it with 10x10x3? The gradients have 5 channels while the input only has 3.

Assume in a convolutional layer's forward pass we have a 10x10x3 image and 5 3x3x3 kernel, then 10x10x3 * 3x3x3x5 has the output of dimensions 8x8x5. Therefore the the gradients fed backwards to this convolutional layer also have the dimensions 8x8x5.

When calculating the derivative of loss w.r.t. kernels, the formula is the convolution $input * \frac{dL}{dZ}$. But if the gradients have dimensions 8x8x5, how is it possible to convolve it with 10x10x3? The gradients have 5 channels while the input only has 3.

Since during the forward pass the kernel window does element-wise multiplication and brings the channels down to 1, do the gradients propagate back to each of the 3 channel equally? Should the 8x8x5 gradients be reshaped into 8x8x1x5 and broadcasted into 8x8x3x5 before convolving with the layer input?