I have read that all the math responsible for modern day machine learning and AI was already in place in 1900s but we did not have computational resources to implement those algorithms. So, is that true? And if it is, in what areas of machine learning the researchers work? And are all the future breakthroughs will be dependent only on increment of computational resources?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ Hey.we are living in the future;the future isn't just about what we can build or engineer basing on Computational Resources. we are making groundbreaking discoveries and achieving remarkable breakthroughs in the field of Artificial Intelligence.So here ML is in!. And we're doing it every week. We'll cover the biggest ones by breaking down complex research papers and analyzing the implications of their content.Therefore,Grid Computing is playing it big and bigger!!..no more worries about resources anymore..and here comes quantum computing! $\endgroup$– quintumniaApr 5, 2017 at 17:58
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
High-level answer: Increase in resources has been important in AI, and definitely was a factor with Deep Blue, but Machine Learning is a newer method that seems to produces more optimal results with less resources on problems of greater complexity.
Here is an article on AlphaGo's hardware: "Google reveals the mysterious custom hardware that powers AlphaGo"
Also an interesting analysis on Quora: "What hardware does AlphaGo run on? Is it customized hardware for best performance?"
Still pretty powerful systems, but I think the algorithms are as important as the computing resources, because all the hardware in the world won't help in the algorithms are poor.
Matthew Lai, creator of Giraffe Chess, said:
"In the ensuing two decades [since Deep Blue], both computer hardware and AI research advanced the state-of-art chess-playing computers to the point where even the best humans today have no realistic chance of defeating a modern chess engine running on a smartphone."
Source: TechExplore
which suggests that hardware and software are both important parts of the overall equation.
$\begingroup$
$\endgroup$
most of the underlying math(like back-propergation) was discovered a long time ago, and the advances in hardware have only recently made it possible to tackle some problems within a reasonable amount of time but that is not the only reason. Some of the other things that also made deep learning pick up includes:
The availability of large, labeled, quality data - deep learning generally requires a lots of data to train successfully. This kind of data just wasn't there until recently.
Algorithmic improvements(I.e further improvements on the maths) - better weight initialization techniques, the discovery of the ReLu as a replacement for the sigmoid and tach activation which goes a long way towards solving the vanishing gradient problem, the discovery of neutral network dropout which is found to reduce overfitting, etc.
The discovery of better neutral network architectures.
So, whist the computing power is certainly a big part of it, it certainly isn't the only factor.
