2
$\begingroup$

I'm still on my first steps in the Data Science field. I played with some DL frameworks, like TensorFlow (pure) and Keras (on top) before, and know a little bit of some "classic machine learning" algorithms like decision trees, k-nearest neighbors, etc.

For example, image classification problems can be solved with deep learning, but some people also use the SVM.

Why are traditional ML models still used over neural networks, if neural networks seem to be superior to traditional ML models? Keras is rather simple to use, so why don't people just use deep neural networks with Keras? What are the pros and cons of each approach (considering the same problem)?

$\endgroup$
  • $\begingroup$ Hi Douglas. I think that this question is a little bit too broad. Maybe you should focus on the comparison of two specific models (e.g. neural networks vs SVMs) to solve the same problem. $\endgroup$ – nbro Jan 14 at 14:01
  • $\begingroup$ think as far I know almost all problems solved with machine learning algorithms can be solved with neural nets correct if I'm wrong. But AI is a really huge field and some defend learn ML before DL others says it's possible dive directly into DL. In a beginner perspective seems DL is much more superior due it's versatility and less verbosity the main point here is why K nearest, classification trees, SVM, SGD is still used over neural nets and if it's worth dive into each one of these techniques. But yes, it could be focused in SVM vs CNN that is the most common example, a good start. $\endgroup$ – Douglas Ferreira Jan 14 at 14:40
  • 1
    $\begingroup$ So, is your question more like "Why are still traditional ML models used over neural networks if neural networks seem to be superior to traditional ML models?"? $\endgroup$ – nbro Jan 14 at 14:44
  • $\begingroup$ Yep, I agree so $\endgroup$ – Douglas Ferreira Jan 14 at 16:07
  • $\begingroup$ I agree with the point of Douglas, both the methods are very broad in itself having their own advantages and limitations. Saying one is superior to others won't justify $\endgroup$ – PureSoftware Jan 15 at 4:16
2
$\begingroup$

This question is very broad, so let me attempt to answer it using my own background in time series analysis.

As an example, why would I continue using ARIMA to forecast a time series? Why not simply use an LSTM model by default, since this is a type of recurrent neural network that takes time-related dependencies into account?

Well, an LSTM model is not good at modelling all time series. It is effective when it comes to modelling volatile data, but ARIMA still outperforms when it comes to forecasting trend data - LSTM tends to overemphasise volatile patterns in future predictions.

Let's take an example of forecasting weekly hotel cancellations by potential customers. The second time series shows much more variability in the number of weekly hotel cancellations than the first:

H1 Time Series

h1 time series

H2 Time Series

h2 time series

Based on MDA (mean directional accuracy), RMSE (root mean squared error), and MFE (mean forecast error) - ARIMA demonstrates superior performance overall for the first time series, while LSTM shows better performance for the second:

performance

On the basis of this example - which is quite specific given the broadness of your question - deep learning techniques are not always used because simpler models can perform better under certain circumstances. It is all about understanding the data you are working with and then choosing the model - not the other way around.

|improve this answer|||||
$\endgroup$
1
$\begingroup$

Why are still traditional machine learning (ML) models used over neural networks if neural networks seem to be superior to traditional ML models?

Of course, the model that achieves state-of-the-art performance depends on the problem, available datasets, etc., so a comprehensive comparison between traditional ML models and deep neural networks is not appropriate for this website, because it requires a lot of time and space. However, there are certain disadvantages of deep neural networks compared to traditional machine learning models, such as k-nearest neighbors, linear regression, logistic regression, naive Bayes, Gaussian processes, support vector machines, hidden Markov models and decision trees.

  • Often, traditional ML models are conceptually simpler (for example, k-NN or linear regression are much simpler than deep neural networks, such as LSTMs).

  • Personally, I've noticed that traditional ML models can be used more easily compared to deep neural networks, given the existence of libraries, like scikit-learn, which really have a simple and intuitive API (even though you apparently do not agree with this).

  • Deep neural networks usually require more data than traditional ML models in order not to overfit. Empirically, I've once observed that certain traditional ML models can achieve comparable performance to deep neural networks in the case of small training datasets.

  • Even though there's already a new and promising area of study called Bayesian deep learning, most deep neural networks do not really provide any uncertainty guarantees, they only provide you a point estimate. This is a big limitation, because, in areas like healthcare, uncertainty measures are required. In those cases, Gaussian processes may be more appropriate.

|improve this answer|||||
$\endgroup$
0
$\begingroup$

SVM is generally considered objectively better than deep learning for standard machine learning tasks.

SVM or decision trees.

Deep Learning is beneficial when there is structure in the data that can't be easily represented by some type of kernel.

I'm actually really interested in why decision trees haven't been used for computer vision in conjunction with deep learning feature extraction.

|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.