Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
I have been coming across visualizations showing that the neural nets tend to perform better as compared to the traditional machine learning algorithms (Linear regression, Log regression, etc.)
Assuming that we have sufficient data to train deep/neural nets, can we ignore the traditional machine learning topics and concentrate more on the neural network architectures?
Given the huge amount of data, are there any instances where traditional algorithms outperform neural nets?
"Assuming that we have sufficient data..." — that's quite a big assumption. Also, traditional methods are well understood, while neural networks (and especially deep learning) is still something of a black box: you train it, and then you get a mapping from input to output. But you don't really know how that mapping is achieved.
It's not only about performance, it's also about efficiency (speed, use of power, etc) and transparency (being able to explain why something happens).
So there are several reasons why we don't put all our eggs into the NN basket:
it is a lot easier to see what's happening and diagnose errors with 'traditional' methods which are well-understood. This is an important point in real-life applications
in many cases we do not have the required amount of training data available that is necessary for deep learning approaches to work
training a DL system is much more time- (and energy) consuming than other algorithms
I'd much rather have a nuclear power plant operated by a traditional algorithm that makes a few mistakes, but nothing drastic (and being aware that it makes these kinds of mistakes allows you to guard against them), than have a total black box doing it where I have no idea why decisions are reached and what happens in edge cases not covered by the training data.
It's fine for toy projects where the stakes are low, but in real-world applications there are often different constraints that DL systems cannot satisfy.
UPDATE: from my own professional experience — working on a conversational AI system for a major bank. Anything they do has to go through layers upon layers of compliance regulation and vetting. Now I'd challenge anyone to explain to a corporate lawyer that your NN will never give unsound advice, and sign in blood on the dotted line that you know exactly under which conditions which advice is given. This is much easier to do with an old-fashioned rule-based system.
