I've been following a deep learning book and the current section I'm on is about convolutional neural networks. The author presents some code to create a basic CNN with about 1 million parameters, which he manages to train to 99.2% accuracy within 12 epochs on the full MNIST dataset.
His output looks like this:
Train on 60000 samples, validate on 10000 samples Epoch 1/12-loss:0.2800 acc:0.9147 val_loss:0.0624 val_acc:0.9794 Epoch 2/12-loss:0.1003 acc:0.9695 val_loss:0.0422 val_acc:0.9854 Epoch 3/12-loss:0.0697 acc:0.9789 val_loss:0.0356 val_acc:0.9880 Epoch 4/12-loss:0.0573 acc:0.9827 val_loss:0.0282 val_acc:0.9910 Epoch 5/12-loss:0.0478 acc:0.9854 val_loss:0.0311 val_acc:0.9901 Epoch 6/12-loss:0.0419 acc:0.9871 val_loss:0.0279 val_acc:0.9908 Epoch 7/12-loss:0.0397 acc:0.9883 val_loss:0.0250 val_acc:0.9914 Epoch 8/12-loss:0.0344 acc:0.9891 val_loss:0.0288 val_acc:0.9910 Epoch 9/12-loss:0.0329 acc:0.9895 val_loss:0.0273 val_acc:0.9916 Epoch 10/12-loss:0.0305 acc:0.9909 val_loss:0.0296 val_acc:0.9904 Epoch 11/12-loss:0.0291 acc:0.9911 val_loss:0.0275 val_acc:0.9920 Epoch 12/12-loss:0.0274 acc:0.9916 val_loss:0.0245 val_acc:0.9916 Test loss: 0.02452171179684301 Test accuracy: 0.9916
Using this code.
Running that same code on my machine, after 12 epochs, I'm barely at 0.6 accuracy. I did have to modify a couple function calls as keras has changed a couple things since the book came out. I'm going crazy trying to figure out why his is so quick. He presents the results as "this is what should happen when the code is run." I understand it had already went through almost 500 gradient descent steps by the time epoch 1 is complete, but is that really enough to reach 98% accuracy right off the bat??