Which problems have neural networks yet to solve?

Question

(Gross Oversimplification) Neural Networks model systems, black boxes with a set of inputs, and a set of outputs. To train a network for modeling this system, obtain hundreds (or millions) of possible inputs/output pairs. This is called the data set, and the network and its optimization algorithm are set to find a set of network parameters that best match the I/O of the network with the I/O of the system.

Are there any systems, for which we have functional data sets, that have yet to be meaningfully modeled with Neural Networks in any form (recurrent, deep, convolutional, etc)?

Answer 1

I haven't seen any dataset where some standard models worked and neural networks utterly failed.

For columnar data (e.g. Excel files / database dumps / CSV files) which contain structured data usually tree-based models like random forests and gradient boosting work better, but neural networks are also usually way better than random.

If you demand other things, e.g. explanations for the decision then Bayesian models might give you an easier time. Or for baselines/simple implementations linear models. Or for real time applications...

Answer 2

If i understand your question correctly , you are asking if there exists functional datasets for which there are no proven solutions based on neural networks which give substantial accuracy.

there are many such problems for which we have data in abundancy, question answering would be one such a thing , you still cant devise a neural network architechture that reads through entire principia mathematica and then complete theorems , and point cloud processing is a also a big hurdle for neural networks considering the highly irregular datastructure , even if you voxelize a point-cloud it would be infeasible to train large convolutional networks on it . (there is also rapid progress in this direction ,, point cloud processing).

geoffrey hinton mentioned in an AMA before 3 years that we will see neural networks that will answer questions based on videos in the next five years , but still video-question answering seems to be far away from the present technology.

Graph datasets are also one such area where still neural networks research is in infancy (refer http://www.inference.vc/how-powerful-are-graph-convolutions-review-of-kipf-welling-2016-2/)