1
$\begingroup$

Given the following data samples (square and triangle mean two classes), why is it suitable to use a Gaussian RBF (radial basis function) in SVM to separate the two classes?

enter image description here

$\endgroup$

1 Answer 1

0
$\begingroup$

If the data is not linearly separable in the input space like your above 2-class case, the Gaussian RBF kernel can map it to a higher-dimensional feature space where a (nonlinear) hyperplane can effectively separate the classes, allowing your SVM to handle non-linear decision boundaries. By computing the kernel matrix $K$ where $K_{i,j}=\exp(-\|\mathbf{x_i}-\mathbf{x_j}\|^2/2\sigma^2)$ contains pairwise similarities between all data points, each data point $\mathbf{x_i}$ is implicitly mapped into a higher-dimensional space defined by the RBF kernel. And the decision boundary in the transformed higher-dimensional feature space is known to be $\sum_i^N\alpha_iy_iK(\mathbf{x_i},\mathbf{x})+b=0$ after applying the kernel trick, where $N$ is the number of support vectors and $y$ is the class label, and of course you need much knowledge of SVM dual optimization and representation theorem to rigorously prove it which your reference perhaps already does so. Obviously in your above case there're only a few support vectors involved, therefore it's quite suitable to use SVM with RBF kernel to classify your data.

RBF kernel also has a hyperparameter $\sigma$ shown above controlling the width of the Gaussian, thus tuning this hyperparameter via methods such as grid search allows your SVM to adapt RBF kernel to the specific characteristics of your training data.

Of course this doesn't mean there're no other suitable methods such as MLP neural networks which could also classify your data albeit perhaps you need all the data.

$\endgroup$
4
  • $\begingroup$ explanation is poor and not convincing. the question why RBF kernel can separate these classes is not answered. what you wrote is a generic knowledge. $\endgroup$ Commented Jan 28 at 7:02
  • $\begingroup$ You judge so quick! I suggest you spend more time to digest and judge, where's your confusion and not convincing? $\endgroup$
    – cinch
    Commented Jan 28 at 7:05
  • $\begingroup$ Your question is also a very generic and simple one, generic SVM/RBF/kernel trick knowledge is enough to address it, maybe you could expand your question to highlight your specific question assuming you already have all the generic knowledge in my answer. Your data only have a few support vectors thus the decision boundary in the kernel space is easy to compute following generic knowledge as already mentioned in my answer, this is the main reason of its suitability imo, you may not even need a computer! $\endgroup$
    – cinch
    Commented Jan 28 at 7:12
  • $\begingroup$ Also I notice most of your questions lack reference or background context, please add more so for your (future) questions to fill the gap of your generic knowledge which others may be completely unaware of. $\endgroup$
    – cinch
    Commented Jan 28 at 7:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .