Why do we need convolutional neural networks instead of feed-forward neural networks?

Question

Why do we need convolutional neural networks instead of feed-forward neural networks?

What is the significance of a CNN? Even a feed-forward neural network will able to solve the image classification problem, then why is the CNN needed?

Answer 1

Why are CNNs useful?

The main property of CNNs that make them more suitable than FFNNs to solve tasks where the inputs are images is that they perform convolutions (or cross-correlations).

Convolution

The convolution is an operation (more precisely, a linear operator) that takes two functions $f$ and $h$ and produces another function $g$. It's often denoted as $f \circledast h = g$, where $\circledast$ represents the convolution operation and $g$ is the function that results from the convolution of the functions $f$ and $h$.

In the case of CNNs,

• $f$ is a multi-dimensional array (aka tensor) and it represents an image (or a processed version of an image, i.e. a feature map)
• $h$ is a multi-dimensional array and it is called kernel (aka filter), which represents the learnable parameters of the CNN, and
• $g$ is a processed version (with $h$) of $f$ and it is often called the feature map, so it's also a multi-dimensional array

Images as functions

To be consistent with the initial definition of the convolution, $f, h$, and $g$ can indeed be represented as functions.

Suppose that the input image is a greyscale (so it is initially represented as a matrix), then we can represent it as a function as follows $$f: [a, b] \times [c, d] \rightarrow [0, 1],$$ i.e. given two numbers $x \in [a, b]$ and $y \in [c, d]$, $f$ outputs a number in the range $[0, 1]$, i.e. $f(x, y) = z$, where $z$ is the grayscale intensity of the pixel at coordinates $x$ and $y$. Similarly, the kernel $h$ and $g$ can also be defined as a function $h: [a, b] \times [c, d] \rightarrow [0, 1]$ and $g: [a, b] \times [c, d] \rightarrow [0, 1]$, respectively.

To be more concrete, if the shape of the image $f$ is $28 \times 28$, then it is represented as the function $f: [0, 28] \times [0, 28] \rightarrow [0, 1]$.

Note that the domain of the images doesn't have to range from $0$ to $28$ and the codomain doesn't have to range from $[0, 1]$. For example, in the case of RGB images, the codomain can also equivalently range from $0$ to $255$.

RGB images can also be represented as functions, more precisely, vector-valued functions, i.e.

$$ f(x, y) = \begin{bmatrix} r(x, y) \\ g(x, y) \\ b(x, y) \end{bmatrix} $$ where

• $r: [a, b] \times [c, d] \rightarrow [0, 1]$ represents the red channel,
• $g: [a, b] \times [c, d] \rightarrow [0, 1]$ represents the green channel, and
• $b: [a, b] \times [c, d] \rightarrow [0, 1]$ represents the blue channel

Or, equivalently, $f: [a, b] \times [c, d] \times [0, 1]^3$.

Why is the convolution useful?

The convolution of an image with kernels (e.g. the median kernel) can be used to perform many operations.

For example, the convolution of a noisy image with the median filter can be used to remove noise from that image.

This is a screenshot of an image from this article, which you should read if you want to understand more about noise removal. So, on the left, there's the noisy image, and, on the right, there's the convolution of the median filter with the noisy image, which removes (at least, partially) the initial noise (i.e. those dots, which are due to the so-called "pepper and salt" noise).

The convolution of any image with the Sobel filter can be used to compute the derivatives of that image (both in the $x$ and $y$ directions, from which you can compute the magnitude and orientation of the gradient at each pixel of the image). See this article for more info.

So, in general, the convolution of an image with a kernel processes the image and the results (i.e. another image, which, in the case of CNNs, is called a feature map) can be different depending on the kernel.

This is the same thing as in CNNs. The only difference is that, in CNNs, the kernels are the learnable (or trainable) parameters, i.e. they change during training so that the overall loss (that the CNN is making) reduces (in the case CNNs are trained with gradient descent and back-propagation). For this reason, people like to say that CNNs are feature extractors or are performing feature extraction (aka feature learning or representation learning).

(Moreover, note that the convolution and cross-correlation are the same operations when the kernels are symmetric (e.g. in the case of a Gaussian kernel). In the case of CNNs, the distinction between convolution and cross-correlation doesn't make much sense because the kernels are learnable. You can ignore this if you are a beginner, but you can find more details here.)

Other useful properties

There are other useful properties of CNNs, most of them are just a consequence of the use of the convolution

• Translation invariance (or equivariance), i.e. they can potentially find the same features (if you think of them as feature extractors) in multiple places of the image independently of their position, orientation, etc. See this answer for more details.

• The equivalent FFNN has a lot more parameters (so CNNs may be less prone to overfitting)

• They often use a sub-sampling operation (known as pooling) to further reduce the number of parameters (which can possibly help to avoid overfitting) and introduce non-linearity.

Notes

Note that the FFNN can also be used to process images. It's just that the CNN is more suited to deal with images for the reasons described above.