Yes, it absolutely makes sense as the Iris dataset is linearly separable (in the sense that linear decision boundaries are near optimal). This can just about be seen in the scatter plots: [![enter image description here][1]][1] Particularly look at petal width versus petal length, two linear decision boundaries for that pair of variables gives a very low error rate already. This shouldn't be surprising as the dataset was introduced in a paper by Fisher describing his linear discriminant method (about 1936?). As @htl (+1) rightly points out, a large neural network has way too much capacity for this problem and will likely overfit. However you will probably find that a small neural network fares no better than a linear classifier for this problem. There is a lot of excitement about deep neural networks over the last few years, but they are no better than most existing classifiers for these sorts of benchmark dataset. They are best (IMHO) for problems with very large datasets, or where convolutional layers are useful. The [comp.ai.neural-nets][2] (an old Usenet news group that we used before the WWW - I trained my first neural net back in 1990) FAQ is well worth a read for anyone wanting to experiment with neural networks. It contains a lot of folk wisdom that is good to know. From [part 3][3]: > Subject: How many hidden layers should I use? > ============================================== > > You may not need any hidden layers at all. Linear and generalized > linear models are useful in a wide variety of applications (McCullagh > and Nelder 1989). And even if the function you want to learn is mildly > nonlinear, you may get better generalization with a simple linear > model than with a complicated nonlinear model if there is too little > data or too much noise to estimate the nonlinearities accurately. This is excellent advice. [1]: https://i.sstatic.net/su21f.png [2]: http://www.faqs.org/faqs/ai-faq/neural-nets/part1/preamble.html [3]: http://www.faqs.org/faqs/ai-faq/neural-nets/part3/