Deep Learning and Neural Networks are getting most of the focus because of recent advances in the field and most experts believe it to be the future of solving machine learning problems.
But make no mistake, classical models still produce exceptional results and in certain problems, they can produce better results than deep learning.
Linear Regression is still by far the most used machine learning algorithm in the world.
It’s difficult to identify a specific domain where classical models always perform better as the accuracy is very much determined on the shape and quality of the input data.
So algorithm and model selection is always a trade-off. It’s a somewhat accurate statement to make that classical models still perform better with smaller data sets. However, a lot of research is going into improving deep learning model performance on less data.
Most classical models require less computational resources so if your goal is speed then its much better.
Also, classical models are easier to implement and visualize which can be another indicator for performance, but it depends on your goals.
If you have unlimited resources, a massive observable data set that is properly labeled and you implement it correctly within the problem domain then deep learning is likely going to give you better results in most cases.
But in my experience, the real-world conditions are never this perfect