1
$\begingroup$

I have to build a model where I pre-process the data with a Gaussian kernel. The data are an $n\times n$ matrix (i.e one channel), but not an image, thus I can't refer to this matrix as an image and to its elements as pixels. The Gaussian kernel is built by the following function (more i.e. here)

$$\begin{equation} \begin{aligned} g(x,y,\sigma) = \dfrac{1}{2\pi\sigma^2} e^{\dfrac{-(x^2+y^2)}{2\sigma^2}}. \end{aligned} \end{equation}$$

This kernel is moving one by one element and doing convolution. In my case, most of the elements are zero, the matrix is sparse.

How can I describe/understand the process of convolving the original data with a Gaussian kernel?

I have been looking for some articles, but I am unable to find any mathematical explanations, only explanation in words or pseudo-code.

$\endgroup$
0
2
$\begingroup$

Mathematically, the convolution is an operation that takes two functions, $f$ and $g$, and produces a third function, $h$. Concisely, we can denote the convolution operation as follows

$$f \circledast g = h$$

In the context of computer vision and, in particular, image processing, the convolution is widely used to apply a so-called kernel (aka filter) to an input (typically, an image, but this does not have to be the case). The input (e.g. an image), the kernel, and the output of the convolution, in this context, is usually a matrix or a tensor. In image processing, the convolution is typically used to e.g. blur images or maybe to remove noise.

However, in the beginning, I said that the convolution is an operation that takes two functions (and not matrices) and produces a third one, so these two explanations of the convolution do not seem to be consistent, right?

The answer to this question is that the two explanations are consistent with each other. More precisely, if you have a function $f : X \rightarrow Y$ (assuming that $X$ is discrete/countable), you can represent it in a vector form as follows $\mathbf{f} = [y_1, y_2, \dots, y_n]$, i.e. $\mathbf{f}$ is a vector that contains all outputs of the function $f$ (for all possible inputs).

In image processing, an image and a kernel can also be thought of as a function with a discrete domain (i.e. the pixels), so the matrices that represent the image or the kernel are just the vector forms of the corresponding functions. See this answer for more details about representing an image as a function.

Once you understand that the convolution in image processing is really the convolution operation as defined in mathematics, then you can simply look up the mathematical definition of the convolution operation.

In the discrete case (i.e. you can think of the function as vectors, as explained above), the convolution is defined as

$${\displaystyle h[n] = (f \circledast g)[n]=\sum _{m=-M}^{M}f[n-m]g[m].} \tag{1}\label{1}$$

You can read equation $1$ as follows

  • $f \circledast g$ is the convolution of the input function (or matrix) $f$ and the kernel $g$
  • $(f \circledast g)[n]$ is the output of the convolution $f \circledast g$ at index (or input position) $n$ (so you need to apply equation \ref{1} for all $n$, if you want to have $h$ and not just $h[n]$)
  • So, the result of the convolution at $n$, $h[n]$, is defined as $\sum _{m=-M}^{M}f[n-m]g[m]$, a sum that goes from $m = -M$ to $m = M$. Here $M$ may be half of the length of the kernel matrix. For example, if you use the following Gaussian kernel, then $M = 2$ (and I assume that the center of the kernel is at coordinate $(0, 0)$).

$$ \mathbf{g} = \frac{1}{273} \begin{bmatrix} 1 & 4 & 7 & 4 & 1 \\ 4 & 16 & 26 & 16 & 4 \\ 7 & 26 & 41 & 26 & 7 \\ 4 & 16 & 26 & 16 & 4 \\ 1 & 4 & 7 & 4 & 1 \end{bmatrix} \label{2}\tag{2} $$

Here are some notes:

  • The kernel \ref{2} is symmetric around the $x$ and $y$ axes: this actually implies that the convolution is equal to the cross-correlation, so you don't even have to worry about their equivalence or not (in case you have ever worried about it, which would have happened only if you already came across the cross-correlation). See this question for more info.

  • The kernel \ref{2} is the vector form of the function form of the 2d Gaussian kernel (the one in your question): more precisely, an integer-valued approximation of the 2D Gaussian kernel when $\sigma = 1$ (as stated in your slides).

  • The convolution can be implemented as matrix multiplication. This may not be useful now, but it's something useful to know if you want to implement it. See this question for more info.

Question for you: what is the result of the application of this Gaussian kernel to any input? What does this kernel intuitively do? Once you fully understand the convolution, you can answer this question.

$\endgroup$
2
  • 1
    $\begingroup$ Thank you for clear answer, this is exactly what I needed to further understand the convolution! So the kernel intuitively multiplies corresponding elements of the input vector with kernel values and count it together, since my matrix is very sparse in most cases and radius arround non-zero element are often only 0s, the result is only multiplication of the one (or few) non-zero element plus 0 times rest. $\endgroup$ – Ruli Sep 30 '20 at 6:04
  • 1
    $\begingroup$ @Ruli Note that if you use a matrix instead of a vector (to represent the input and kernel), you will need 2 sums (one that goes horizontally across the kernel and image and one that goes vertically) in the definition of the discrete convolution (rather than just 1, like I wrote above, which is the definition for 1-dimensional signals, i.e. vectors). Maybe this Wikipedia article will clarify more what I mean. Maybe I will edit my answer later to be more precise and include this info. $\endgroup$ – nbro Sep 30 '20 at 13:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.