0
$\begingroup$

I have build a model where I pre-process data with Gaussian kernel. The data are $n\times n$ matrix of one channel, but not an image, thus I cant reference to this matrix as image and to its elements as pixels. The Gaussian kernel is build by following function (more i.e. here)

$$\begin{equation} \begin{aligned} g(x,y,\sigma) = \dfrac{1}{2\pi\sigma^2} e^{\dfrac{-(x^2+y^2)}{2\sigma^2}}. \end{aligned} \end{equation}$$

This kernel is moving one by one element and doing convolution. In my case, most of the elements are zero, the matrix is sparse. How can I describe the process of convolving the original data itself and producing output? I have been looking for some articles but am unable to find any mathematical explanations, only explanation in words or pseudo-code.

$\endgroup$
  • $\begingroup$ Are you asking for the mathematical formulation of the convolution in general? Or are you asking how you would convolve, specifically, the 2d Gaussian kernel with a 2-dimensional matrix? $\endgroup$ – nbro Sep 29 at 16:15
  • $\begingroup$ for understanding general definition would be good too, but I need the 2D gaussian convolution specifically, if there is a difference $\endgroup$ – Ruli Sep 29 at 16:51
  • $\begingroup$ The convolution operation is the same even if you change your kernel. The only thing that changes is what you convolve (i.e. the input and the kernel), and so also the result. I've seen that in the slides that you're linking us to there is the definition of the 2d Gaussian kernel. If nobody provides an answer meanwhile, I think you can find online some step-by-step examples that show you how the convolution is applied, and you can just replace the kernel that they use in those examples with the Gaussian kernel. $\endgroup$ – nbro Sep 29 at 17:02
2
$\begingroup$

Mathematically, the convolution is an operation that takes two functions, $f$ and $g$, and produces a third function, $h$. Concisely, we can denote the convolution operation as follows

$$f \circledast g = h$$

In the context of computer vision and, in particular, image processing, the convolution is widely used to apply a so-called kernel (aka filter) to an input (typically, an image, but this does not have to be the case). The input (e.g. an image), the kernel, and the output of the convolution, in this context, is usually a matrix or a tensor. In image processing, the convolution is typically used to e.g. blur images or maybe to remove noise.

However, in the beginning, I said that the convolution is an operation that takes two functions (and not matrices) and produces a third one, so these two explanations of the convolution do not seem to be consistent, right?

The answer to this question is that the two explanations are consistent with each other. More precisely, if you have a function $f : X \rightarrow Y$ (assuming that $X$ is discrete/countable), you can represent it in a vector form as follows $\mathbf{f} = [y_1, y_2, \dots, y_n]$, i.e. $\mathbf{f}$ is a vector that contains all outputs of the function $f$ (for all possible inputs).

In image processing, an image and a kernel can also be thought of as a function with a discrete domain (i.e. the pixels), so the matrices that represent the image or the kernel are just the vector forms of the corresponding functions. See this answer for more details about representing an image as a function.

Once you understand that the convolution in image processing is really the convolution operation as defined in mathematics, then you can simply look up the mathematical definition of the convolution operation.

In the discrete case (i.e. you can think of the function as vectors, as explained above), the convolution is defined as

$${\displaystyle h[n] = (f \circledast g)[n]=\sum _{m=-M}^{M}f[n-m]g[m].} \tag{1}\label{1}$$

You can read equation $1$ as follows

  • $f \circledast g$ is the convolution of the input function (or matrix) $f$ and the kernel $g$
  • $(f \circledast g)[n]$ is the output of the convolution $f \circledast g$ at index (or input position) $n$ (so you need to apply equation \ref{1} for all $n$, if you want to have $h$ and not just $h[n]$)
  • So, the result of the convolution at $n$, $h[n]$, is defined as $\sum _{m=-M}^{M}f[n-m]g[m]$, a sum that goes from $m = -M$ to $m = M$. Here $M$ may be half of the length of the kernel matrix. For example, if you use the following Gaussian kernel, then $M = 2$ (and I assume that the center of the kernel is at coordinate $(0, 0)$).

$$ \mathbf{g} = \frac{1}{273} \begin{bmatrix} 1 & 4 & 7 & 4 & 1 \\ 4 & 16 & 26 & 16 & 4 \\ 7 & 26 & 41 & 26 & 7 \\ 4 & 16 & 26 & 16 & 4 \\ 1 & 4 & 7 & 4 & 1 \end{bmatrix} \label{2}\tag{2} $$

Here are some notes:

  • The kernel \ref{2} is symmetric around the $x$ and $y$ axes: this actually implies that the convolution is equal to the cross-correlation, so you don't even have to worry about their equivalence or not (in case you have ever worried about it, which would have happened only if you already came across the cross-correlation). See this question for more info.

  • The kernel \ref{2} is the vector form of the function form of the 2d Gaussian kernel (the one in your question): more precisely, an integer-valued approximation of the 2D Gaussian kernel when $\sigma = 1$ (as stated in your slides).

  • The convolution can be implemented as matrix multiplication. This may not be useful now, but it's something useful to know if you want to implement it. See this question for more info.

Question for you: what is the result of the application of this Gaussian kernel to any input? What does this kernel intuitively do? Once you fully understand the convolution, you can answer this question.

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Thank you for clear answer, this is exactly what I needed to further understand the convolution! So the kernel intuitively multiplies corresponding elements of the input vector with kernel values and count it together, since my matrix is very sparse in most cases and radius arround non-zero element are often only 0s, the result is only multiplication of the one (or few) non-zero element plus 0 times rest. $\endgroup$ – Ruli Sep 30 at 6:04
  • 1
    $\begingroup$ @Ruli Note that if you use a matrix instead of a vector (to represent the input and kernel), you will need 2 sums (one that goes horizontally across the kernel and image and one that goes vertically) in the definition of the discrete convolution (rather than just 1, like I wrote above, which is the definition for 1-dimensional signals, i.e. vectors). Maybe this Wikipedia article will clarify more what I mean. Maybe I will edit my answer later to be more precise and include this info. $\endgroup$ – nbro Sep 30 at 13:08
  • $\begingroup$ of course that is logical :) $\endgroup$ – Ruli Sep 30 at 13:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.