# Are filters fixed or learned?

No matter what I google or what paper I read, I can't find an answer to my question. In a deep convolutional neural network, let's say AlexNet (Krizhevsky, 2012), filters' weights are learned by means of back-prop.

But how are kernels themselves selected? I know kernels had been used in image processing long before CNNs, hence I'd imagine there would be a set of filters based on kernels (see, for example, this article) that are proven to be effective for edge detection and the likes.

Reading around the web, I also found something about "randomly generated kernels". Does anyone know if and when this practice is adopted?

### What are filters in image processing?

In the context of image processing (and, in general, signal processing), the kernels (also known as filters) are used to perform some specific operation on the image. For example, you can use a Gaussian filter to smooth the image (including its edges).

### What are filters in CNNs?

In the context of convolutional neural networks (CNNs), the filters (or kernels) are the learnable parameters of the model.

Before training, the kernels are usually randomly initialised (so they are not usually hardcoded). During training, depending on the loss or error of the network (according to the loss function), the kernels (or filters) are updated, so that to minimise the loss (or error). After training, they are typically fixed. Incredibly, the filters learned by CNNs can be similar to the Gabor filter (which is thought to be related to our visual system [1]). See figure 9.19 of chapter 9 (p. 365) of the Deep Learning book by Goodfellow et al.

The number of kernels that are applied to a given input (and more than one kernel is often applied) in a CNN is a hyper-parameter.

### What are the differences and similarities?

In both contexts, the words "kernel" and "filter" are roughly synonymous, so they are often used interchangeably. Furthermore, in both cases, the kernels are related to the convolution (or cross-correlation) operation. More specifically, the application of a filter, which is a function $$h$$, to an input, which is another function $$f$$, is equivalent to the convolution of $$f$$ and $$h$$. In mathematics, this is often denoted by $$f \circledast h = g$$, where $$\circledast$$ is the convolution operator and $$g$$ is the result of the convolution operation and is often called the convolution (of $$f$$ and $$h$$). In the case of image processing, $$g$$ is the filtered image. In the case of CNNs, $$g$$ is often called an activation map.