Can someone explain to me the difference between machine learning and deep learning? Is it possible to learn deep learning without knowing machine learning?
Deep learning is a specific variety of a specific type of machine learning. So it's possible to learn about deep learning without learning all of machine learning, but it requires learning some machine learning (because it is some machine learning).
Machine learning refers to any technique that focuses on teaching the machine how it can learn statistical parameters from a large amount of training data. One particular type of machine learning is artificial neural networks, which learn a network of nonlinear transformations that can approximate very complicated functions of wide arrays of input variables. Recent advances in artificial neural networks have to do with how to train deep neural networks, which have more layers than normal and also special structure to deal with the challenges of learning more layers.
Deep learning is one form of machine learning. Learning with decision trees is another form.
Deep learning refers to learning with deep neural networks. Neural networks are one group of many that can be used to learn from data:
- Neural Networks
- Decision Trees and Random Forests
- Support Vector Machines
- Bayesian Approaches
- k-nearest neighbors
As acceptable as the inclusions represented by the above Venn diagram may be, it is neither remarkably telling nor accurate.
The terms Artificial Intelligence, Machine Learning, and Deep Learning, although not brands are not precisely scientific either. They are umbrella terms named by people with particular funding, communications, and departmental perspectives at the time they published the terms that stuck.
Furthermore, the order of things is incorrect. What is called artificial intelligence today was a centuries-old goal of using machines to automate mental activities that, in those days, required human attention and perhaps extend human mental abilities through that automation. This vision arose long before switching circuitry and information theory and therefore before computer science.
Consequently, computer science is actually a subset of that AI vision and subsequent to it.
Deep learning is a term based on the precarious and oversimplified idea that there is a correlation between the number of layers in an artificial network and the depth of abstraction the layer can achieve. Since the count of activation cells in a layer had been called the layer's width, the selection of depth was chosen to express the dimension of layer quantity. This is odd because typical diagramming depicts the number of cells per layer as height, the number of layers as width, and the diagram has no depth because it is 2-D. But that's not the real issue. There is no scientific basis for the depth of a network and the depth of abstraction, just a hunch, and there is considerable evidence in the field of computer vision that it is not that simple.
This next Venn diagram is neither authoritative nor perfectly representative either, but some of the issues with the above one are repaired. Even though it answers the question more intelligently in several respects, the problems in word choice in jargon are usually never corrected without a significant push, and this post won't nearly be enough.
When I started Machine Leraning chapters in book used to look like this
- Linear models
- Logestic Regression
- Neural Network
- Decision Tress and Random Forest
- Boosting and Bagging
- SVD and SVM
II) UnSupervised Learning:
- Gaussian Mixture Model
- DB Scan
III) ReInforment Learning:
All of a sudden chapter I>2>b created a sub-field of its own . Well to know why, let me tell you a bit of history.
Machine learning word was coined in 1959 by Arthur Samuel to signify that
machines were able to learn from data than explicit instruction. Initally it was broken into two groups based on if th approach required label data or not(ie regression, classification), then they realised we can cassify by clustering too which gave birth to unsupervised. And word reinforment learning was born inspired by areas of game theory. Lets keep those details aside for later.
Coming to deep learnign, the word
deep learning came very recently, as recent as 2008 from a Geoff Hinton conference. There people started using it to indicate a very deep neural network architecture used in a paper presented by Geoff Hinton and from then onwards it kind of became as a new way of classifying machine learning besides
reinforcement.(Disc: There may be odd reference of calling NN as DL before this but not so popular and acceptable prior to this)
Well I sometimes feel the name
deep learning is somewhat misnomer, it would have been better of if it was named as
neural learning or to stress on depth maybe
deep neural learning. If you are new you might be wondering what depth I am talking about, the entire word deep came from the fact that neural network (thanks the availability of high processing abilities of GPUs) were now able to train successfully on multiple layers. The word deep can also be loosely used to include other non-neural network areas of machine learning which requires lots of computation like
deep belief net or
recurrent net. To be precise the units of the networks today are no longer a mere
neuron or a
perceptron, it can be
GRU or a
capsule, so I guess word
deep now makes more sense than before.