What are the variables used in a Gaussian radial basis kernel in the context of SVMs?

If I have the Gaussian kernel

$$k(x, x') = \operatorname{exp}\left( -\| x - x' \|^2 / 2\sigma^2 \right)$$

What is $$x$$ and $$x'$$ in the context of training an SVM?

$$\mathbf{x} \in \mathbb{R}^p$$ and $$\mathbf{x}' \in \mathbb{R}^p$$ are two inputs (or feature vectors).
In the context of classification with an SVM, you are given a dataset $$D = \{(\mathbf{x}_i, y_i) \}_{i=1}^N$$, where $$\mathbf{x}_i \in \mathbb{R}^p$$ is an input (or point) and $$y_i$$ the corresponding label. The goal is to find a hyperplane that classifies the points $$\mathbf{x}_i$$. The hyperplane actually corresponds to a binary classifier that splits the plane into two, so the assumption is that there are two labels. However, these points $$\mathbf{x}_i$$ may not be linearly separable in $$\mathbb{R}^p$$, i.e. there may not be a hyperplane (in 2d, i.e. when $$p=2$$, a hyperplane is a line) that separates them. The kernel trick, i.e. the use of kernels (such as the Gaussian radial basis), allows an SVM to perform non-linear classification by transforming the inputs to a space where they are linearly separable.
• In your example, $\mathbf{x}$ (or $\mathbf{x}'$) would be a MNIST image, which can be represented as a vector in $\mathbb{R}^{784}$ (where $784 = 28*28$). In your example, you would have $500$ examples of the form $\mathbf{x}$. The labels don't have to be a vector, they can simply be a scalar, so your labels could simply be a vector of the form $\mathbb{R}^{500}$. Regarding how you apply the kernel, this is more a detail of how the SVM works, but, to give you the intuition, essentially, you will be replacing the dot products in the SVM computations with $k(\mathbf{x}, \mathbf{x}')$. – nbro Mar 1 '20 at 15:59
• @FeedMeInformation Yes. (But the application of a kernel function $k$ is not just restricted to the data points (image vectors), but, given that it is a general function, it can be applied to any pair of vectors. You can ignore this, if you don't want to know the details of SVMs. But, if you read the Wikipedia entry on SVMs, you will get a better feeling of the whole picture (even though Wikipedia may not be the most reliable source ever)). – nbro Mar 1 '20 at 18:09