3
$\begingroup$

This article suggests that deep learning is not designed to produce the universal algorithm and cannot be used to create such a complex systems.

First of all it requires huge amounts of computing power, time and effort to train the algorithm the right way and adding extra layers doesn't really help to solve complex problems which cannot be easily predicted.

Secondly some tasks are extremely difficult or impossible to solve using DNN, like solving a math equations, predicting pseudo-random lists, fluid mechanics, guessing encryption algorithms, or decompiling unknown formats, because there is no simple mapping between input and output.

So I'm asking, are there any alternative learning algorithms as powerful as deep architectures for general purpose problem solving? Which can solve more variety of problems, than "deep" architectures cannot?

$\endgroup$
  • $\begingroup$ I've updated with few examples. Universal one which doesn't require desired fixed mapping, because not all problems can be solved by finding the right mapping, on the other hand it's not my requirement, as I don't want to limit my question to only these. Universal that can for example guess encoding of some file formats, but it's again, I don't want to limit my question to anything particular technology. $\endgroup$ – kenorb Aug 5 '16 at 18:34
  • $\begingroup$ This question assumes that Deep architectures are good at "general purpose problem solving" tasks like learning Math, or guessing encryption algorithms. As far as I know, Deep NNs learn feature representations for input, while all these problems operate at a higher level than that. $\endgroup$ – Harsh Aug 6 '16 at 1:35
  • $\begingroup$ @Harsh Thanks, clarified that I meant opposite, that DNN aren't good for such tasks, that's why the question is about other architectures which are better at it. $\endgroup$ – kenorb Aug 6 '16 at 1:39
  • $\begingroup$ @kenorb this question needs to be completely rewritten. In particular, the title does not mean what you just said it means. Even then, I would vote to close this question as too broad. $\endgroup$ – Harsh Aug 6 '16 at 1:42
  • $\begingroup$ In short, I'm saying that DNN aren't good for certain tasks, and aren't so universal, so I'm asking whether there are other powerful architectures which can solve tasks which DNN can't. $\endgroup$ – kenorb Aug 6 '16 at 1:43
6
$\begingroup$

Have you read the book The Master Algorithm: by Pedro Domingos?

He discusses the present day machine learning algorithms... Their strengths, weaknesses and applications...

  • Deep Neural Network
  • Genetic Algorithm
  • Bayesian Network
  • Support Vector Machine
  • Inverse Deduction

enter image description here

$\endgroup$
  • 3
    $\begingroup$ Nice, informative picture. $\endgroup$ – NietzscheanAI Aug 5 '16 at 18:45
  • 1
    $\begingroup$ I will follow up on the book recommendation. It looks like a captivating read $\endgroup$ – Seth Simba Jan 4 '18 at 10:37
4
$\begingroup$

Deep learning is actually pretty useful (relative to other techniques) precisely when there is no simple mapping between input and output, and features from the raw input need to be aggregated and combined in complex ways by successive layers to form the output.

As I pointed out in my answer to the AI SE decompilation question, there is recent DL research which takes a natural language description as input and generates program text as output. Despite working in this general research area, I was personally surprised by this - the problem is significantly harder than the 'AI math' link you provide above.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.