1
$\begingroup$

In the formula to calculate output shape of tensor after convolution operation $$ W_2 = (W_1-F+2P)/S + 1, $$ where:

  • $W_2$ is the output shape of the tensor
  • $W_1$ is the input shape
  • $F$ is the filter size
  • $P$ is the padding
  • $S$ is the stride.

Why do we add $1$? It gets us to the correct answer, but how is this formula derived?

Source: https://cs231n.github.io/convolutional-networks/#pool

$\endgroup$
0
0
$\begingroup$

In a few words, we add $1$ to account for the initial position of the kernel. You can easily see this if you let $s = 1$ (unit stride) and $p = 0$ (i.e. no padding), so your formula simplifies to

\begin{align} W_2 = (W_1 - F) + 1, \label{1}\tag{1} \end{align}

So, in this simplified case and, for simplicity, assuming squared inputs and kernels, the width (or height) of the output of the convolutional layer is the number of steps that we slide the kernel horizontally (or vertically, respectively), for example, starting from the top left of the input, plus the initial position of the kernel. In this case, $(W_1 - F)$ is the number of times we slide the kernel horizontally (or vertically) and $+1$ is to account for the initial position of the kernel.

Here's an animation of this simplified case for $W_1 = 4$ and $F = 3$ (the green matrix is the output, while the blue one is the input).

enter image description here

So, if we apply the formula \ref{1}, we should get $W_2 = 2$, as we can see from the animation above, which produces a $2 \times 2$ matrix. In fact, $(4 - 3) + 1 = 2 = W_2$. You can see from the animation that we slide the kernel only once in each of the axes, but the shape of the output matrix is $2 \times 2$ because we compute the dot product between the kernel and the submatrix of the input that corresponds to the initial position of the kernel.

A detailed explanation of this (section 2.4, relationship 6) and other simpler formulas to compute the size of the output of a convolutional layer can be found in the report A guide to convolution arithmetic for deep learning, which has also many images that illustrate the concepts (you can find animations here too).

$\endgroup$
1
  • 1
    $\begingroup$ Thank you, exactly answers my doubt! $\endgroup$ Aug 5 at 16:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.