14
votes
What are the domains where SVMs are still state-of-the-art?
State-of-the-art is a tough bar, because it's not clear how it should be measured. An alternative criteria, which is akin to state-of-the-art, is to ask when you might prefer to try an SVM.
SVMs have ...
- 9,207
7
votes
What are the domains where SVMs are still state-of-the-art?
Deep Learning and Neural Networks are getting most of the focus because of recent advances in the field and most experts believe it to be the future of solving machine learning problems.
But make no ...
- 171
6
votes
Accepted
What is a support vector machine?
I find the chapter on machine learning from Russell & Norvig is a pretty good place to start with SVMs. I think this is Chapter 18?
One way to understand an SVM is as a kind of neural network, ...
- 9,207
6
votes
Are the shortcomings of neural networks diminishing?
Just to add to what has been said in @MikeWise's brilliant answer,
All things equal, deep learning models generally rank supreme when compared to other algorithms as the size of the dataset increases:...
- 840
6
votes
Accepted
Are the shortcomings of neural networks diminishing?
Neural Networks have other short comings as well.
It takes much longer and far more resources to train a neural network than something like a random forest. So if you need speed of training or are ...
- 176
6
votes
Why does training an SVM take so long? How can I speed it up?
The most likely explanation is that you're using too many training examples for your SVM implementation.
SVMs are based around a kernel function. Most implementations explicitly store this as an NxN ...
- 9,207
6
votes
Do Support Vector Machines have the ability to learn while in use?
Support vector machines are a supervised machine learning method that can be used for both classification and regression problems. They are based on finding a Maximum Margin hyperplane that best ...
- 1,094
3
votes
How do I use a taxonomy and the support vector machine for question classification?
This is not an answer (I don't have enough reputation to comment). I did something close to this in my master's thesis and think it is close to what you are interested in.
In it, I had developed a ...
- 161
3
votes
What is the definition of the hinge loss function?
The hinge loss/error function is the typical loss function used for binary classification (but it can also be extended to multi-class classification) in the context of support vector machines, ...
- 39.6k
2
votes
Why would LDA have performed much better than SVM and Naive Bayes in diagnosing ADHD?
It would be hard to tell if you don't provide what kind of data/problem you are working on, but LDA works well when data that are grouped in gaussian blobs surrounding centroids while vanilla SVM ...
- 121
2
votes
What is the purpose of the "gamma" parameter in SVMs?
I've summarized the key ideas of SVMs. So this is how $\gamma$ is used with a gaussian Kernel:
$$K_{\text{Gauss}}(\mathbf{x}_i, \mathbf{x}_j) = e^{\frac{-\gamma\|\mathbf{x}_i - \mathbf{x}_j\|^2}{2 \...
- 1,055
2
votes
Accepted
Is an SVM the same as a neural network without a hidden layer?
First, what makes the neural network different than linear regression is the non-linearity (activation function), not the number of layers. So, a neural network with $n$ layers with no non-linearities ...
- 242
2
votes
How do you perform a gradient based adversarial attack on an SVM based model?
A way to avoid computing the SVM loss by hand is to use a differentiable programming framework, such as JAX. These frameworks will automatically calculate gradients using automatic differentiation.
If ...
- 276
2
votes
Accepted
How do I combine models trained on different data to increase classification accuracy?
You can try using a multi-input model. Here is a recent post with a similar discussion, with the required architecture defined in the answer.
Instead of combining the separate models, you can create ...
- 890
2
votes
Accepted
What is the definition of the "cost" function in the SVM's objective function?
That is the hinge loss, a type of loss most notably used for SVM classification. The hinge loss is typically defined as:
$$
\ell(y)=\max (0,1-t \cdot y),
$$
which, in your use case, is something like ...
- 1,098
2
votes
Accepted
What are support values in a support vector machine?
In the least-squares SVM (LS-SVM) the non-zero Lagrange multipliers ($\alpha$) are the support values. The corresponding data points are the support vectors. Johan Suykens explains this in Least ...
- 1,863
2
votes
Accepted
Does the image is logistic regression or SVM, and why?
The straight dashed-line shows the typical decision line in logistic regression or any linear classifier. The dashed-circle shows the decision line from SVM.
Obviously, since the data is not linearly ...
- 86
2
votes
Accepted
Why doesn't the set $\{ -2, +2 \}$ in $E(X) = (y − \text{sign}\{\overline{W} \cdot \overline{X} \}) \in \{ −2, +2 \}$ include $0$?
It is important to note that the exact statement is the eqation given below can never be 0 for misclassified points in $ S^+$
$$ E(X) = (y - \text{sign}\{\overline{W} \cdot \overline{X}\}) $$
And $S+$ ...
- 61
2
votes
Accepted
What is the smoothness assumption in SVMs?
Smoothness here is the mathematical definition, so as you implied smoothness is ruled out by output data with sharp spikes or discontinuous jumps (and possibly the data of the gradient, the gradient's ...
- 36
2
votes
Distinguishing text with opposite meanings in SVM (False Information Detection)
Going step by step:
Preprocessing
Preprocessing is a big deal in NLP, out there you'll find many tutorials describing the classic steps but few explanations about why and when you should actually ...
- 5,218
1
vote
Support Vector Machine Convert optimisation problem from argmax to argmin
So actually I managed to get hold of my lecturer to explain the argmax to argmin conversion.
Generally speaking maximising $\frac{1}{||w||}$ is identical to minimising $||w||$. As $||w||$ in $\frac{1}{...
1
vote
Accepted
Why does the training time of SVMs dramatically decrease after applying dimensionality reduction to the features?
SVM complexity is $O(\max(n,d)\min(n,d)^2)$ according to Chapelle, Olivier. "Training a support vector machine in the primal." Neural Computation 19.5 (2007): 1155-1178.
$n$ is the number of ...
- 249
1
vote
Is my 57% sports betting accuracy correct?
My question is, can I rely on my Accuracy (mean & standard deviation) for future games even though my Testing Accuracy is lower than 52.5%?
If by Accuracy you mean training accuracy, then ...
- 30.3k
1
vote
Accepted
How to understand mapping function of kernel?
A kernel function $f : \mathcal{X} \times \mathcal{X} \rightarrow \mathbb{R}$ is a valid support vector kernel if it is a Mercer kernel. Mercer's condition essentially ensures that the Gram matrix of ...
- 86
1
vote
How to implement SVM algorithm from scratch in a programming language?
Based on this repository:
https://github.com/arkm97/svm-from-scratch/blob/master/SVM_from_scratch.ipynb
I will try to reverse engineering that concept:
So firstly there is issue for DataCleaning(...
- 144
1
vote
Why do we use the word "kernel" in the expression "Gaussian kernel"?
The usage of the word "kernel" in the context of support vector machines probably comes from its usage in the context of integral transforms.
See the article Kernel of an integral operator, and the ...
- 39.6k
1
vote
Accepted
What are the variables used in a Gaussian radial basis kernel in the context of SVMs?
$\mathbf{x} \in \mathbb{R}^p$ and $\mathbf{x}' \in \mathbb{R}^p$ are two inputs (or feature vectors).
In the context of classification with an SVM, you are given a dataset $D = \{(\mathbf{x}_i, y_i) ...
- 39.6k
1
vote
Does the bag-of-visual-words method improve the classification accuracy?
Bag-of-visual words (BOVW) was classicly used in computer vision before the introduction of neural networks or some more advanced classical techniques, such us VLAD or Fisher Vectors. In any case, it ...
- 21
1
vote
How does an svm work? How does it perform comparisons between malignant and benign tumor
I will try to give you a simplified explanation of how SVMs work.
The data one works with can be of two types. Either it is very easily separable and there is a clear straight line boundary between ...
- 163
Only top scored, non community-wiki answers of a minimum length are eligible