14 votes

What are the domains where SVMs are still state-of-the-art?

State-of-the-art is a tough bar, because it's not clear how it should be measured. An alternative criteria, which is akin to state-of-the-art, is to ask when you might prefer to try an SVM. SVMs have ...
7 votes
Accepted

Are the shortcomings of neural networks diminishing?

Neural Networks have other short comings as well. It takes much longer and far more resources to train a neural network than something like a random forest. So if you need speed of training or are ...
  • 186
7 votes

What are the domains where SVMs are still state-of-the-art?

Deep Learning and Neural Networks are getting most of the focus because of recent advances in the field and most experts believe it to be the future of solving machine learning problems. But make no ...
6 votes
Accepted

What is a support vector machine?

I find the chapter on machine learning from Russell & Norvig is a pretty good place to start with SVMs. I think this is Chapter 18? One way to understand an SVM is as a kind of neural network, ...
6 votes

Are the shortcomings of neural networks diminishing?

Just to add to what has been said in @MikeWise's brilliant answer, All things equal, deep learning models generally rank supreme when compared to other algorithms as the size of the dataset increases:...
6 votes

Why does training an SVM take so long? How can I speed it up?

The most likely explanation is that you're using too many training examples for your SVM implementation. SVMs are based around a kernel function. Most implementations explicitly store this as an NxN ...
3 votes

How do I use a taxonomy and the support vector machine for question classification?

This is not an answer (I don't have enough reputation to comment). I did something close to this in my master's thesis and think it is close to what you are interested in. In it, I had developed a ...
  • 161
3 votes

What is the definition of the hinge loss function?

The hinge loss/error function is the typical loss function used for binary classification (but it can also be extended to multi-class classification) in the context of support vector machines, ...
  • 35.5k
2 votes

Why would LDA have performed much better than SVM and Naive Bayes in diagnosing ADHD?

It would be hard to tell if you don't provide what kind of data/problem you are working on, but LDA works well when data that are grouped in gaussian blobs surrounding centroids while vanilla SVM ...
  • 121
2 votes

What is the purpose of the "gamma" parameter in SVMs?

I've summarized the key ideas of SVMs. So this is how $\gamma$ is used with a gaussian Kernel: $$K_{\text{Gauss}}(\mathbf{x}_i, \mathbf{x}_j) = e^{\frac{-\gamma\|\mathbf{x}_i - \mathbf{x}_j\|^2}{2 \...
  • 1,015
2 votes
Accepted

Is an SVM the same as a neural network without a hidden layer?

First, what makes the neural network different than linear regression is the non-linearity (activation function), not the number of layers. So, a neural network with $n$ layers with no non-linearities ...
  • 242
2 votes
Accepted

How do I combine models trained on different data to increase classification accuracy?

You can try using a multi-input model. Here is a recent post with a similar discussion, with the required architecture defined in the answer. Instead of combining the separate models, you can create ...
  • 880
2 votes
Accepted

What is the definition of the "cost" function in the SVM's objective function?

That is the hinge loss, a type of loss most notably used for SVM classification. The hinge loss is typically defined as: $$ \ell(y)=\max (0,1-t \cdot y), $$ which, in your use case, is something like ...
  • 1,048
2 votes
Accepted

What are support values in a support vector machine?

In the least-squares SVM (LS-SVM) the non-zero Lagrange multipliers ($\alpha$) are the support values. The corresponding data points are the support vectors. Johan Suykens explains this in Least ...
2 votes
Accepted

Does the image is logistic regression or SVM, and why?

The straight dashed-line shows the typical decision line in logistic regression or any linear classifier. The dashed-circle shows the decision line from SVM. Obviously, since the data is not linearly ...
  • 76
2 votes
Accepted

Why doesn't the set $\{ -2, +2 \}$ in $E(X) = (y − \text{sign}\{\overline{W} \cdot \overline{X} \}) \in \{ −2, +2 \}$ include $0$?

It is important to note that the exact statement is the eqation given below can never be 0 for misclassified points in $ S^+$ $$ E(X) = (y - \text{sign}\{\overline{W} \cdot \overline{X}\}) $$ And $S+$ ...
  • 61
2 votes
Accepted

What is the smoothness assumption in SVMs?

Smoothness here is the mathematical definition, so as you implied smoothness is ruled out by output data with sharp spikes or discontinuous jumps (and possibly the data of the gradient, the gradient's ...
2 votes

Distinguishing text with opposite meanings in SVM (False Information Detection)

Going step by step: Preprocessing Preprocessing is a big deal in NLP, out there you'll find many tutorials describing the classic steps but few explanations about why and when you should actually ...
1 vote

Support Vector Machine Convert optimisation problem from argmax to argmin

So actually I managed to get hold of my lecturer to explain the argmax to argmin conversion. Generally speaking maximising $\frac{1}{||w||}$ is identical to minimising $||w||$. As $||w||$ in $\frac{1}{...
1 vote
Accepted

Why does the training time of SVMs dramatically decrease after applying dimensionality reduction to the features?

SVM complexity is $O(\max(n,d)\min(n,d)^2)$ according to Chapelle, Olivier. "Training a support vector machine in the primal." Neural Computation 19.5 (2007): 1155-1178. $n$ is the number of ...
1 vote

Is my 57% sports betting accuracy correct?

My question is, can I rely on my Accuracy (mean & standard deviation) for future games even though my Testing Accuracy is lower than 52.5%? If by Accuracy you mean training accuracy, then ...
1 vote
Accepted

How to understand mapping function of kernel?

A kernel function $f : \mathcal{X} \times \mathcal{X} \rightarrow \mathbb{R}$ is a valid support vector kernel if it is a Mercer kernel. Mercer's condition essentially ensures that the Gram matrix of ...
  • 86
1 vote

How to implement SVM algorithm from scratch in a programming language?

Based on this repository: https://github.com/arkm97/svm-from-scratch/blob/master/SVM_from_scratch.ipynb I will try to reverse engineering that concept: So firstly there is issue for DataCleaning(...
1 vote

Why do we use the word "kernel" in the expression "Gaussian kernel"?

The usage of the word "kernel" in the context of support vector machines probably comes from its usage in the context of integral transforms. See the article Kernel of an integral operator, and the ...
  • 35.5k
1 vote

How do you perform a gradient based adversarial attack on an SVM based model?

A way to avoid computing the SVM loss by hand is to use a differentiable programming framework, such as JAX. These frameworks will automatically calculate gradients using automatic differentiation. ...
1 vote
Accepted

What are the variables used in a Gaussian radial basis kernel in the context of SVMs?

$\mathbf{x} \in \mathbb{R}^p$ and $\mathbf{x}' \in \mathbb{R}^p$ are two inputs (or feature vectors). In the context of classification with an SVM, you are given a dataset $D = \{(\mathbf{x}_i, y_i) ...
  • 35.5k
1 vote

Does the bag-of-visual-words method improve the classification accuracy?

Bag-of-visual words (BOVW) was classicly used in computer vision before the introduction of neural networks or some more advanced classical techniques, such us VLAD or Fisher Vectors. In any case, it ...
1 vote

How does an svm work? How does it perform comparisons between malignant and benign tumor

I will try to give you a simplified explanation of how SVMs work. The data one works with can be of two types. Either it is very easily separable and there is a clear straight line boundary between ...
  • 163

Only top scored, non community-wiki answers of a minimum length are eligible