6
$\begingroup$

I've been reading a lot about DL. I can understand to an extent how it works, in theory at least, and how it's technically different from conventional ML. But what I'm looking for is more of a "conceptual" meaning.

Let's say you're designing a self-learning system, why would you choose DL? What are the main performance advantages that DL offers? Is it the accuracy, speed, power efficiency, a mix of all of them?

$\endgroup$
6
$\begingroup$

Deep learning allows you to solve complex problems without necessarily being able to specify the important "features" or key input variables for the model in advance.

To give an example, a problem that may be easily tackled without deep learning could be predicting the frequency and claim amounts of insurance vehicle claims, given historical claim data that may include various attributes of the policy holder and their vehicle. In this example, the "features" to be specified in the model are the known attributes of policy holder and vehicle. The model will then attempt to utilise these features to make predictions.

On the other hand, facial recognition is a problem more suited to deep learning algorithms. This is because it is difficult to manually identify what combinations of pixels may be important features to include in a conventional machine learning model. A multi-layered neural network however has the potential to identify/create the important features itself, which may include for example eyes, nose and mouth, and then utilise these features to recognise faces and other objects.

$\endgroup$
  • $\begingroup$ thanks a lot. So when we compare a specific DL algorithm to a non-DL algorithm, can we say it's better because it's more accurate? Or because it made solving the problem possible? Also if we compare 2 DL algorithms, what parameters do we prioritize? Accuracy or speed? Or perhaps other factors, say like memory or power efficiency? Or perhaps even a mix of different parameters for the specific application? $\endgroup$ – Mahmoud Abdel-Mon'em Mar 8 '17 at 12:50
5
$\begingroup$

Deep Learning these days mean a lot of things to a lot of people, its quickly becoming a buzz-word. But so far it still retains two very important conceptual properties:

Does away with most feature engineering work. This was mentioned in the answer above, but this is very important. It really saves a lot of work.

Allows you to make maximal use of unlabelled data. This is strictly speaking available to other approaches, not just Deep Learning, but its in DL that this really took off. And typically labelled data is very hard to get while unlabelled is all over the place. Things like denoising autoencoders and Restricted Boltzmann machines are just wonderful.

$\endgroup$
2
$\begingroup$

Deep learning allows you to not know the answer in order to ask the program a question. Their main benefit is their finite ability and flexible nature.

The problem with procedural programing to solve problems is you have to know what the computer needs to do in order to solve the problem.

What deep learning does is remove the requirement of the programmer to know how to solve the problem by having them only need to know what the computer needs to know.

This is the entire premise of neural networks. The programmer writes the program for data points required to be known in order to solve a particular problem.

The computer is given an input it comes up with an answer. If it's answer is wrong it needs to make the answer it gave less likely and the right answer more likely. The goal is to get the computer to always get the right answer. If the computer always gets the wrong answer then the neural network it too small.

What deep learning is, is a neural network that is deep. To answer this you need to know how a neural network is.

A neural network is based on a neuron:

  • Finite number of `boolean' inputs (More then one)
  • A weight is attached on each input to define how important often though as a float between -1 and 1, but it's just a percentage of how likely each input changes the answer.
  • One boolean output

A neuron can be a class or function the implementation really doesn't matter. The weight of each input changes as more answers are asked and responses verified.

The depth of a neural network is has one layer when there is one row of neurons between the input and output. two layers when a few neurons make decisions on inputs and a final neuron or multiple neurons make decisions biased on those neurons.

A neural network is called deep when there are at least four layers of neurons? (do some research don't take my word for it ^_^)

The disadvantage of deep learning is that it's ability is finite. There is no way a deep neural network by it's self to get smarter then it's programed to be. It has a intelligence curve similar to root time if it isn't improved somehow. This leads to the other problem in neural networks. While the programmer has no need to know how decisions are made by the computer they still need to know what questions or nodes need to be added. The reason this is a problem is if the nodes responsible aren't there the program will be wrong in strange cases and have no way of correcting this on it's own. The larger the network the harder it is to solve these kinds of problems.

This will lead to an inevitable solution to have the computer self improve by some type of generative algorithm. This has it's own breadth of problems as if not built properly could grow into something unintended which wastes time and money if it fails quickly, and could be potentially dangerous if it appears to work and doesn't.

The answer to AI will be a combination of deep neural networks some generative type programing and some new ideas and innovations.

$\endgroup$
1
$\begingroup$

Shallow layered networks are less capable of recursive or extended abstraction necessary for the kinds of generalization needed to handle complex tasks common in real world applications.

It is the same problem as was discovered nearly a century ago in the analog world. One can try to reduce the components in the old tube radio design to lower its cost, but tuning and amplification had a minimal number of independent operations required. After decades, the basic functions are integrated into one wafer of silicon, yet no single transistor can accomplish the entire task. The more complex the externality controlled, the more sophisticated the control system, whether or not it is a learning system.

In the basic NN architecture most familiar to those in machine learning, in general, width is driven by degrees of freedom in the input and output regions. Depth is driven by the need to approximate nonlinear control complexities.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.