Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
I've been reading a lot about DL. I can understand to an extent how it works, in theory at least, and how it's technically different from conventional ML. But what I'm looking for is more of a "conceptual" meaning.
Let's say you're designing a self-learning system, why would you choose DL? What are the main performance advantages that DL offers? Is it the accuracy, speed, power efficiency, a mix of all of them?
Deep learning allows you to solve complex problems without necessarily being able to specify the important "features" or key input variables for the model in advance.
To give an example, a problem that may be easily tackled without deep learning could be predicting the frequency and claim amounts of insurance vehicle claims, given historical claim data that may include various attributes of the policy holder and their vehicle. In this example, the "features" to be specified in the model are the known attributes of policy holder and vehicle. The model will then attempt to utilise these features to make predictions.
On the other hand, facial recognition is a problem more suited to deep learning algorithms. This is because it is difficult to manually identify what combinations of pixels may be important features to include in a conventional machine learning model. A multi-layered neural network however has the potential to identify/create the important features itself, which may include for example eyes, nose and mouth, and then utilise these features to recognise faces and other objects.
Deep Learning these days mean a lot of things to a lot of people, its quickly becoming a buzz-word. But so far it still retains two very important conceptual properties:
Does away with most feature engineering work. This was mentioned in the answer above, but this is very important. It really saves a lot of work.
Allows you to make maximal use of unlabelled data. This is strictly speaking available to other approaches, not just Deep Learning, but its in DL that this really took off. And typically labelled data is very hard to get while unlabelled is all over the place. Things like denoising autoencoders and Restricted Boltzmann machines are just wonderful.
Deep learning allows you to not know the answer in order to ask the program a question. Their main benefit is their finite ability and flexible nature.
The problem with procedural programing to solve problems is you have to know what the computer needs to do in order to solve the problem.
What deep learning does is remove the requirement of the programmer to know how to solve the problem by having them only need to know what the computer needs to know.
This is the entire premise of neural networks. The programmer writes the program for data points required to be known in order to solve a particular problem.
The computer is given an input it comes up with an answer. If it's answer is wrong it needs to make the answer it gave less likely and the right answer more likely. The goal is to get the computer to always get the right answer. If the computer always gets the wrong answer then the neural network it too small.
What deep learning is, is a neural network that is deep. To answer this you need to know how a neural network is.
A neural network is based on a neuron:
A neuron can be a class or function the implementation really doesn't matter. The weight of each input changes as more answers are asked and responses verified.
The depth of a neural network is has one layer when there is one row of neurons between the input and output. two layers when a few neurons make decisions on inputs and a final neuron or multiple neurons make decisions biased on those neurons.
A neural network is called deep when there are at least four layers of neurons? (do some research don't take my word for it ^_^)
The disadvantage of deep learning is that it's ability is finite. There is no way a deep neural network by it's self to get smarter then it's programed to be. It has a intelligence curve similar to root time if it isn't improved somehow. This leads to the other problem in neural networks. While the programmer has no need to know how decisions are made by the computer they still need to know what questions or nodes need to be added. The reason this is a problem is if the nodes responsible aren't there the program will be wrong in strange cases and have no way of correcting this on it's own. The larger the network the harder it is to solve these kinds of problems.
This will lead to an inevitable solution to have the computer self improve by some type of generative algorithm. This has it's own breadth of problems as if not built properly could grow into something unintended which wastes time and money if it fails quickly, and could be potentially dangerous if it appears to work and doesn't.
The answer to AI will be a combination of deep neural networks some generative type programing and some new ideas and innovations.
Shallow layered networks are less capable of recursive or extended abstraction necessary for the kinds of generalization needed to handle complex tasks common in real world applications.
It is the same problem as was discovered nearly a century ago in the analog world. One can try to reduce the components in the old tube radio design to lower its cost, but tuning and amplification had a minimal number of independent operations required. After decades, the basic functions are integrated into one wafer of silicon, yet no single transistor can accomplish the entire task. The more complex the externality controlled, the more sophisticated the control system, whether or not it is a learning system.
In the basic NN architecture most familiar to those in machine learning, in general, width is driven by degrees of freedom in the input and output regions. Depth is driven by the need to approximate nonlinear control complexities.
