When is deep learning overkill?

Question

For example, for classifying emails as spam, is it worthwhile - from a time/accuracy perspective - to apply deep learning (if possible) instead of another machine learning algorithm? Will deep learning make other machine learning algorithms like naive Bayes unnecessary?

Answer 1

It's all about Return On Investment. If DL is "worth doing", it's not overkill.

If the cost of using DL (computer cycles, storage, training time) is acceptable, and the data available to train it is plentiful, and if the marginal advantage over alternative algorithms is valuable, then DL is a win.

But, as you suggest, if your problem is amenable to alternate methods, especially if it offers a signal that matches up well with classic methods like regression or naive Bayes, or your problem requires explanation of why the decision boundary is where it is (e.g. decision trees), or if your data lacks the continuous gradients needed by DL (especially, CNNs), or your data varies over time which would require periodic retraining (especially, at unpredictable intervals), then DL probably is a mismatch for you.

Answer 2

Deep learning is powerful but it is not a superior method than bayesian. They work well in what they are designed to do:

Use deep learning:

• Cost for computation is much cheaper than cost of sampling (e.g: natural language processing)
• If you have highly non-linear problem
• If you want to simplify feature engineering
• If you don't have prior distribution (e.g: setting the weights to random Gaussian). Or you do but you don't mind the complexity.
• If you want accuracy for speed (deep learning is slow)

Use naive bayesian:

• If you have prior distribution that you want to use
• If you want to update your model quickly and easily (in particular conjour models)
• If you have your own likelihood function and wish to "control" how exactly the model works
• If you want to model hierarchial models
• If you don't want to tweak parameters
• If you want a faster model, both in training and execution
• If you want to make the independence assumption
• If you want to prevent overfitting (that's a very simple model)