Questions tagged [support-vector-machine]

For questions about support vector machines (SVMs), which are supervised learning models with associated learning algorithms that analyze data used for classification and regression analysis.

Filter by
Sorted by
Tagged with
1 vote
1 answer
22 views

Why are SVMs / Softmax classifiers considered linear while neural networks are non-linear?

My understanding is that neural networks are definitely not linear classifiers, as the point of functions like ReLU is to introduce non-linearity. However, here's where my understanding starts to ...
user avatar
  • 121
1 vote
0 answers
39 views

Why does the SVM perform poorly on test data that has a different class distribution than the training data?

Do you know why the SVM performs poorly on test data that has a different class distribution than the training data? The training data has around 15 classes, and the additional testing data has around ...
user avatar
  • 11
0 votes
0 answers
9 views

Stopping criteria for SMO when no solution exists for Hard Margin SVM

When we solve the dual problem using SMO, we pick two $\mu_i$'s at a time and optimize the dual wrt them while satisfying the required constraints. But in case we are using the no slack formulation of ...
user avatar
0 votes
0 answers
23 views

In traditional R-CNN, why do we use an SVM after training a ConvNet with a log loss?

I'm learning about development of object detection algorithms and came across this fact which seems strange to me - In the multi-stage pipeline of R-CNN, after using a region extractor to crop the ...
user avatar
0 votes
0 answers
17 views

Appropriate machine learning approach for object detection of connector

I am suppose to detect a female connector which happens to be an automotive part. I need to draw a bounding box around the connector when it appear. Here's the closest resemblance part I could find to ...
user avatar
0 votes
1 answer
30 views

Distinguishing text with opposite meanings in SVM (False Information Detection)

I am currently working on a Binary Text Classification Model (False Information Detection) using Support Vector Machine and used TF-IDF as text vectorizer in Python. I have already tried training the ...
user avatar
0 votes
0 answers
16 views

Is this the correct method for how to implement a stratified K fold with grid search SVC?

I'm writing a support vector classifier for a binary class using some toy data. This is the code: ...
user avatar
0 votes
1 answer
61 views

Does $(\langle w, x \rangle + b) = ||x - x'||$ hold?

Currently, I am trying to understand the mathematics of SVM's using the textbook 'Mathematics for Machine Learning' by Deisenrot et. al. On page 375, they define the distance between a point $x$ and ...
user avatar
0 votes
1 answer
68 views

Do I need to tune the hyper-parameters or more data if SVR model performs poorly?

I am using non-linear data to SVR and have tried tuning the hyperparameters and still have a poor model performance. Do I need more data or format the data for more suitable results? I get similar ...
user avatar
0 votes
0 answers
32 views

How to proof that the square of the half-margin of maximum-margin SVM is equal to the summation of lagrange multipliers given by SVM dual?

Can be show that the square of the half-margin of maximum-margin SVM is equal to the summation of Lagrange multipliers given by SVM dual? i.e, Let $ \rho = \dfrac{1}{w}\\ $ Show that: $$ \dfrac{1}{\...
user avatar
0 votes
0 answers
16 views

Weight for Samples on SVM (Support Vector Machine)

There is a option sample_weight in fit(X[, y, sample_weight]) function (OneClassSVM, sklearn library). If I use the option ...
user avatar
0 votes
0 answers
23 views

How can I weight each point in one-class SVM?

I want to give weights to some data points Specifically, these are points related to anomalies (I'm implementing one-class SVM for anomaly detection) Exactly, I want to consider some data points that ...
user avatar
1 vote
0 answers
11 views

Can you correlate decision boundary of final layer of a neural network to predictive distribution?

I was reading in a On the Decision Boundary of Deep Neural Networks that the final layer of a MLP can be equated to an SVM and can generate decision boundaries similar to methods with SVM. I was ...
user avatar
2 votes
1 answer
2k views

What is the definition of the hinge loss function?

I came across the hinge loss function for training a neural network model, but I did not know the analytical form for the same. I can write the mean squared error loss function (which is more often ...
user avatar
  • 3,099
1 vote
1 answer
194 views

What is the smoothness assumption in SVMs?

In this research paper, we have the following claim the smoothness assumption that underlies many kernel methods such as Support Vector Machines (SVMs) does not hold for deep neural networks trained ...
user avatar
  • 159
3 votes
0 answers
82 views

Why is the margin attained with $\Phi=\left[2 x, 2 x^{2}\right]^{T}$ greater than the margin attained with $\Phi=\left[x, x^{2}\right]^{T}$?

I am trying to understand the solution to part 4 of problem 3 from the midterm exam 6.867 Machine learning: Mid-term exam (October 15, 2003). For reproducibility, here is problem 3. We consider here ...
user avatar
1 vote
2 answers
161 views

If the training data are linearly separable, which of the following $L(w)$ has less optimum answer for $w$, when $y = w^Tx$?

I'm studying machine learning and I came into a challenging question. The answer is 2. But based on my ML notes, all of them are true. Where are the wrong points?
user avatar
2 votes
1 answer
154 views

What are support values in a support vector machine?

I started reading up on SVM and very little is defined of what are support values. I reckon it's they are denoted as $\alpha$ in most formulations.
user avatar
2 votes
0 answers
46 views

How does the support vector machine constraint imply that sample selection bias will not systematically affect the output of the optimisation?

I am currently studying the paper Learning and Evaluating Classifiers under Sample Selection Bias by Bianca Zadrozny. In section 3.4. Support vector machines, the author says the following: 3.4. ...
user avatar
1 vote
1 answer
60 views

Does the image is logistic regression or SVM, and why? [closed]

Does the image is logistic regression or SVM, and why?
user avatar
5 votes
1 answer
388 views

How should we interpret this figure that relates the perceptron criterion and the hinge loss?

I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. Chapter 1.2.1.2 Relationship with Support Vector Machines says the following: The perceptron criterion is ...
user avatar
2 votes
1 answer
109 views

Why doesn't the set $\{ -2, +2 \}$ in $E(X) = (y − \text{sign}\{\overline{W} \cdot \overline{X} \}) \in \{ −2, +2 \}$ include $0$?

I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. Chapter 1.2.1.2 Relationship with Support Vector Machines says the following: The perceptron criterion is ...
user avatar
2 votes
1 answer
62 views

Support Vector Machine Convert optimisation problem from argmax to argmin

I'm new to the AI Stackexchange and wasn't certain if this should go here or to Maths instead but thought the context with ML may be useful to understand my problem. I hope posting this question here ...
user avatar
  • 21
1 vote
1 answer
99 views

Why does the training time of SVMs dramatically decrease after applying dimensionality reduction to the features?

Training an SVM with an RBF kernel model with c = 5.5 and gamma = 1.06, for a 5-class classification problem on the NSL-KDD ...
user avatar
1 vote
1 answer
134 views

What is the definition of the "cost" function in the SVM's objective function?

In a course that I am attending, the cost function of a support vector machine is given by $$J(\theta)=\sum_{i=1}^{m} y^{(i)} \operatorname{cost}_{1}\left(\theta^{T} x^{(i)}\right)+\left(1-y^{(i)}\...
user avatar
1 vote
1 answer
229 views

Is my 57% sports betting accuracy correct?

I have been creating sports betting algorithms for many years using Microsoft access and I am transitioning to the ML world and trying to get a grasp on determining the success of my algorithms. I ...
user avatar
0 votes
1 answer
412 views

Which kind of data does sigmoid kernel performance well?

While I was playing with some hyperparameters, I came to a wired situation. My dataset is IRIS dataset to be specific. SVM algorithm has some hyperparameters that we can tune, such as Kernels, and C ...
user avatar
1 vote
1 answer
58 views

How to understand mapping function of kernel?

For a kernel function, we have two conditions one is that it should be symmetric which is easy to understand intuitively because dot products are symmetric as well and our kernel should also follow ...
user avatar
1 vote
0 answers
43 views

Why is the accuracy of my model very low on a separate dataset from the training and test datasets?

I am working on stock price prediction project, I am using the support vector regression (SVR) model for it. As I am splitting my data into train and test, I am getting high accuracy while predicting ...
user avatar
0 votes
1 answer
200 views

Is an SVM the same as a neural network without a hidden layer?

A neural network without a hidden layer is the same as just linear regression. If I then use squared hinge loss and encoporate the l2 regularisation term, is it fair to then call this network the ...
user avatar
1 vote
0 answers
77 views

What is the gradient of a non-linear SVM with respect to the input?

The objective function of an SVM is the following: $$J(\mathbf{w}, b)=C \sum_{i=1}^{m} \max \left(0,1-y^{(i)}\left(\mathbf{w}^{t} \cdot \mathbf{x}^{(i)}+b\right)\right)+\frac{1}{2} \mathbf{w}^{t} \...
user avatar
1 vote
0 answers
36 views

Why is my SVM not reaching good accuracy when trained to perform binary classification of search results?

I am trying to perform binary classification of search results based on the relevance to the query. I followed this tutorial on how to make an SVM, and I got it to work with a small iris dataset. Now, ...
user avatar
  • 116
0 votes
1 answer
204 views

How do you perform a gradient based adversarial attack on an SVM based model?

I have an SVM currently and want to perform a gradient based attack on it similar to FGSM discussed in Explaining And Harnessing Adversarial Examples. I am struggling to actually calculate the ...
user avatar
1 vote
1 answer
47 views

What are the variables used in a Gaussian radial basis kernel in the context of SVMs?

If I have the Gaussian kernel $$ k(x, x') = \operatorname{exp}\left( -\| x - x' \|^2 / 2\sigma^2 \right) $$ What is $x$ and $x'$ in the context of training an SVM?
user avatar
3 votes
1 answer
124 views

How do I poison an SVM with manifold regularization?

I'm working on Adversarial Machine Learning, and have read multiple papers on this topic, some of them are mentioned as follows: Poisoning Attacks on SVMs: https://arxiv.org/pdf/1206.6389.pdf ...
user avatar
2 votes
1 answer
69 views

Does the bag-of-visual-words method improve the classification accuracy?

I'm a beginner in computer vision. I want to know which option among the following two can get better accuracy of image classification. SIFT features + SVM Bag-of-visual-words features + SVM Here's ...
user avatar
1 vote
0 answers
40 views

Scoring feature vector with Support Vector Machine

I am reading the R-CNN paper by Ross Girshick1 et al. (link) and I fail to understand how they do the inference. This is described in the section 2.2.Test-time Detection in the paper. I quote: At ...
user avatar
  • 1,018
1 vote
0 answers
30 views

Is it possible to combine multiple SVMs that were trained on sublayers of a CNN into one combined SVM?

I have created a CNN for use on the MNIST dataset for now (so I have 10 classes). I have trained SVMs on the sublayers of this trained CNN and wish to combine them into a combined SVM as to give a ...
user avatar
1 vote
0 answers
45 views

A generalized quadratic loss and Newton iteration for Support Vector Regression, why doesn't it generalize well?

I'm comparing the results of an Newton optimizer for a modified version of SVM ( a generalized quadratic loss, similar to the one stated in: A generalized quadratic loss for SVM ) with classic SVM^...
user avatar
5 votes
1 answer
975 views

How do I combine models trained on different data to increase classification accuracy?

I have two trained models. One is using a LinearSVC algorithm and is trained on numerical data from medical examination from patients with diabetic retinopathy. The second one is a neural network ...
user avatar
4 votes
1 answer
409 views

How to implement SVM algorithm from scratch in a programming language?

I'm a computer scientist who's studying support vector machines (SVMs) in a machine learning course. I have some understanding of how SVMs are designed, thanks to 16. Learning: Support Vector Machines ...
user avatar
  • 141
1 vote
0 answers
24 views

why my regression model predict every datapoint to the same value

I am trying to train a SVR but I found that with some combination of features, the trained SVR predict every point in test set to the same value. this problem occurs much more when I use linear kernel ...
user avatar
3 votes
1 answer
145 views

Why do we use the word "kernel" in the expression "Gaussian kernel"?

I've heard the expression "Gaussian kernel" in several contexts (e.g. in the kernel trick used in SVM). A Gaussian kernel usually refers to a Gaussian function (that is, a function similar to the ...
user avatar
  • 33.8k
11 votes
4 answers
540 views

What are the domains where SVMs are still state-of-the-art?

It seems that deep neural networks and other neural network based models are dominating many current areas like computer vision, object classification, reinforcement learning, etc. Are there domains ...
user avatar
2 votes
0 answers
33 views

What is the benefit of scaling the hyperparameter C of an SVM?

Please read the following page of the Sklearn documentation. The figure shown there (see below) illustrates why C should be scaled when using a SVM with 'l1' penalty, whereas it shouldn't be scaled ...
user avatar
  • 141
1 vote
1 answer
62 views

How does an svm work? How does it perform comparisons between malignant and benign tumor

How do Support Vector Machines (SVMs) differentiate between a glass and a bottle or between a malignant and a benign tumor when it dealing with it for the first time? What will be the analysis ...
user avatar
2 votes
2 answers
685 views

What is the purpose of the "gamma" parameter in SVMs?

I want to understand what the gamma parameter does in an SVM. According to this page. Intuitively, the gamma parameter defines ...
user avatar
  • 121
9 votes
3 answers
346 views

What is a support vector machine?

What is a support vector machine (SVM)? Is an SVM a kind of a neural network, meaning it has nodes and weights, etc.? What is it best used for? Where I can find information about these?
user avatar
8 votes
7 answers
17k views

Why does training an SVM take so long? How can I speed it up?

I'm trying to create and test non-linear SVMs with various kernels (RBF, Sigmoid, Polynomial) in scikit-learn, to create a model which can classify anomalies and benign behaviors. My dataset includes ...
user avatar
4 votes
2 answers
120 views

Why would LDA have performed much better than SVM and Naive Bayes in diagnosing ADHD?

In a final project in diagnosing Attention deficit hyperactivity disorder (ADHD) using Machine Learning, we obtained parameters from real patients. We used this data and got much higher success rates ...
user avatar
  • 41