I am attempting to train a network to do something I thought would be a relatively simple case to learn with: identify whether the back of a scanned vintage postcard has one of 'no postage stamp', a '1 cent stamp', or a '2 cent stamp.' The images are 250px by about 150px, RGB color, and there are about two thousand of them. Ballpark 75% of them are no-stamp, 20% 1-cent, and 10% 2-cent.

When I attempt to train the network it seems like it is starting at 70 +/- 1 % accurate and hovers in that range for 50 epochs, never improving. I'm not sure I'm reading the metrics correctly, though, as this doesn't seem quite right.

I set this up by following the tutorial on the Keras blog: https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html

I haven't implemented the latter part of the tutorial, where a pre-trained network is used, because I haven't found one that seems like it would be a similar problem.

My training and validation sets are here: https://drive.google.com/open?id=1-TxEKVVvP7RuFC7kFgH7Wt5A8z8QGTR3

And my Google Colab Jupyter notebook is here: https://colab.research.google.com/drive/1UuKDF1wDwYlXszB2ahIrygRnfcs2D_sD

  • $\begingroup$ Machine learning unfortunately doesn't necessarily get better with more training data. If the classes in your problem are not clearly separable for the algorithm, then no amount of data will improve it. One issue could be that the classes are unbalanced; perhaps try to use fewer 'no-stamp' images. Or investigate pre-processing them, eg to amplify colours or other features, so that the classifier can pick them up better. $\endgroup$ – Oliver Mason Mar 18 '19 at 9:30
  • $\begingroup$ Thanks @OliverMason, I will try that. I was also thinking that maybe I need to make sure the ImageGenerator() isn't cropping the image such that the stamp area is missing. $\endgroup$ – pr3sidentspence Mar 18 '19 at 14:47
  • $\begingroup$ I forgot that I already did that, re: ImageGenerator(). :\ $\endgroup$ – pr3sidentspence Mar 18 '19 at 16:18

The answer (as detailed quite thoroughly here and here) is that specifying

model.compile(optimizer = 'rmsprop', loss = 'binary_crossentropy', metrics=['accuracy'])

causes keras to guess, incorrectly, that because I am using binary_crossentropy for the loss function, that I would want to use binary_accuracy as the way of reporting the accuracy metrics. Apparently, one should specify that one wants the categorical_accuracy metrics if one, as I do, has more than two classes.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.