What is a support vector machine (SVM)? Is an SVM a kind of a neural network, meaning it has nodes and weights, etc.? What is it best used for?
Where I can find information about these?
Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.Sign up to join this community
I find the chapter on machine learning from Russell & Norvig is a pretty good place to start with SVMs. I think this is Chapter 18?
One way to understand an SVM is as a kind of neural network, but this is not usually an intuitive approach for a beginner (unless your NN knowledge is already quite good).
A better way to understand SVMs is as consisting of three simple ideas rolled into one algorithm. Here's an attempt at a "For Dummies" answer though:
Maximum Margin Classification. SVMs are usually used to find a pattern in a set of data. Often, the data allow an infinite set of possible patterns that are all equally descriptive. For example, maybe The relationship is "Lives within 5 miles of a Coast -> Income High". It's easy to imagine that this pattern is just as good as "Lives within 5.0001 miles of a Coast -> Income High" or "Lives within 4.999 miles of a Coast -> Income High". There might actually be a lot more play than that in the data (e.g. 3 miles might work out too). If all these are equally good, then the maximum margin idea says you should pick the one that's "in the middle" of the data. So maybe all values between 5.5 and 4.8 are equally good. In that case, we might pick 5.15 (in the middle). This example is super simplified. Real world data would have a lot more variables, and the idea of "in the middle" ends up being a little more complex, but this is the intuition. It turns out that finding the maximum margin pattern is easy when the patterns are linear. That is, when they can be represented by drawing straight lines through a plot of the dataset.
Projection into higher dimensions. This one needs a bit of math to visualize. Consider a dataset consisting of a circular pattern (for instance, maybe the pattern is that higher incomes are found in the middle of the city). There is no linear relationship that captures this pattern. That is, you can't draw a straight line through the data, and say something meaningful about all the values on one side or the other. However, if you add a new feature to your data that is the square of the original coordinates, it's easy to find such a pattern. Basically, if you pre-compute "circular" functions of the original data, you can add them to the dataset, and then find a pattern that is a linear function of this new feature. This idea generalizes: if you compute a complex enough function of your original data, and then apply the maximum margin idea, you can learn any pattern you like. The problem is that it's slow: adding more features makes it take longer to find the patterns you want.
The Kernel Trick. The thing that made SVMs useful was the kernel trick: finding the maximum margin didn't depend on anything except the product of the coordinates of the various points. It turned out that this product could be computed first, and then run through certain functions to produce a problem that was identical to the one you'd get by first adding extra features and then doing the multiplication. However, computing the problem this way didn't require adding any new features! This made SVMs one of the first reliable, well understood, and fast methods for finding non-linear patterns in data.
Hope that provides a starting point. Consider reading Russell & Norvig as a next starting point, or Bishop if you want to go deeper.
In case this is still relevant to you I can share my tutorial on SVM with Python implementation in Jupyter notebook:
The tutorial assumes some mathematics and programming background knowledge. The SVM codes utilize no external machine learning packages and tries to teach the reader to build a SVM model him-/herself.
I hope it helps you!
Support vector machines are supervised learning models with associated learning algorithms that analyze data and are used for classification and regression analysis.
Here is a link where you can learn more about it from a introduction level: "Support Vector Machine — Introduction to Machine Learning Algorithms" (Medium)