6
$\begingroup$

I'm trying to create and test non-linear SVMs with various kernels (RBF, Sigmoid, Polynomial) in scikit-learn, to create a model which can classify anomalies and benign behaviors.

My dataset includes 692703 records and I use a 75/25% training/testing split. Also, I use various combinations of features whose dimensionality is between 1 and 14 features. However, the training processes of the various SVMs take much too long. Is this reasonable? I have also examined the ensemble BaggingClassifier in combination with non-linear SVMs, by configuring the n_jobs parameter to -1; nevertheless, the training process proceeds again too slowly.

How can I speed up the training processes?

Thanks in advance

$\endgroup$
  • 3
    $\begingroup$ Can you show some code and timing? What do you have for computing power? Have you gone through the following thread? stackoverflow.com/questions/31681373/… $\endgroup$ – Brian O'Donnell Jul 20 '18 at 2:16
  • $\begingroup$ This is my opinion and from my experience: you can use integers for speeding up your calculation and training data test with your data frame. $\endgroup$ – Danny Lukmana Apr 21 at 8:17
2
$\begingroup$

The most likely explanation is that you're using too many training examples for your SVM implementation.

SVMs are based around a kernel function. Most implementations explicitly store this as an NxN matrix of distances between the training points to avoid computing entries over and over again.

In your case, with 75% of 700,000 examples, this matrix will require approximately 250GB of RAM to store, which is more than you're likely to have in consumer hardware.

If your SVM implementation can avoid caching the values, you might get a speedup that way, or you might not (you'll waste a lot of time recomputing them).

A much better way to deal with this is to just not use all of the data, since most of it will be redundant from the SVM's perspective (it only benefits from having more data near the decision boundaries). A good starting place would be to randomly discard 90% of the training data, and see what performance looks like.

| improve this answer | |
$\endgroup$
1
$\begingroup$

I think you should use a linear kernel, 'cause training SVM with a linear kernel is faster than with another kernel, especially for text classification. Good luck

https://www.svm-tutorial.com/2014/10/svm-linear-kernel-good-text-classification/

| improve this answer | |
$\endgroup$
1
$\begingroup$

To quickly train the SVM , you can try to Use Linear SVM or Use scaled data.

sources: https://www.researchgate.net/publication/2926909_A_Practical_Guide_to_Support_Vector_Classification_Chih-Wei_Hsu_Chih-Chung_Chang_and_Chih-Jen_Lin

| improve this answer | |
$\endgroup$
  • $\begingroup$ Hi, just so you know, these links may 404 after a while, so if you can, just copy the important part and quote in your answer. $\endgroup$ – Anshuman Kumar Apr 22 at 9:15
0
$\begingroup$

Because you use various combinations of features whose dimensionality is between 1 and 14 features, You might try to use Linear SVM (linear Kernels) would be good for your problem. You could try LIBLINEAR library but the Data should be linearly separable, otherwise test accuracy would be very low.

| improve this answer | |
$\endgroup$
0
$\begingroup$

You can speed up the training time by doing several steps:

  1. scale the values of your features
  2. use only a limited number of features because this will affect the training time; i.e. when you use 14 features, it means your model has 14 dimensions and it makes computation more complex and take much time.
  3. choose a proper kernel, linear SVM kernel usually give the fastest result
| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.