Why didn't my convolutional image classifier network learn anything?

Question

So I am trying to make a CNN image classifier that has two classes, good and bad. The aim is to look at photoshoot pictures that can be found on fashion sites and find the "best one". I trained this model for 150 epochs and its loss did not change at all(Roughly). Details are as follows:

My metric for "best one", also the way I structured my dataset, is that the photo shows the whole outfit or whole body of the model not only like upper body or lower. I also labeled the photos where the model's back was turned to the camera. My training set has 1304 good photos and 2000 bad photos. My validation set has 300 photos per class.(so 600)

Architechture as follows : Conv > Pool > Conv > Pool > Flatten > Linear > Linear > Softmax. For details of architecture like stride etc. check out the code I provided.

I have to use softmax since in my application I need to see the probabilities of being good and bad. That is why I am not using cross-entropy loss but instead using negative log-likelihood loss. Adam is my optimizer

Other hyperparameters: batch size: 64, number of epochs: 150, input size: (224, 224), number of classes: 2, learning rate: 0.01, weight decay: 0.01

I trained with this script for 150 epochs. The model initialized with a 0.5253 loss and ended with a 0.5324 loss. I took snapshots every 10 epochs but I did not learn anything through the learning. This is what my learning curve looks like:

Now I know that there are many many things I can do to make the model perform better like initializing with a pretrained, doing some more stuff with transforms, etc. But the problem with this model is not that it is performing poorly, it is not performing at all! Like, I also have a validation accuracy check and it is around %50 during all training, for a classifier with 2 classes. So I am assuming I am doing something very obvious wrong, any idea what?

Answer 1

Here are some points I noticed:

1. Your data isn't enough for training a deep learning model from scratch. Like you mentioned using a pre-trained model is probably a better alternative like vgg

2. 150 epochs is too much for the amount of data you have it's bound to overfit (however this isn't what will fix your problem here but something to consider for the future)

3. Try augmenting your data. In other words, you can increase your dataset size by randomly rotating images, flipping them, etc. check this link out to learn more

4. Try out smaller learning rates maybe this is why your model isn't learning e.g try out lr=1e-3 or even less to see if that impacts your learning rate. Generally try to play around with your hyperparameters like batch size etc

5. If you didn't preprocess your images you should e.g turning them to grayscale, normalise the data (values ranging from 0 to 1 instead of 0 to 255), or maybe apply other image transformations to denoise your data and make it easier for your model to extract valuable information

6. Try using dropout layers and try playing around with the number units in your dense layers and filters in your convolutional layers. This seems to me to be more of an issue with your data more than anything but you can always play around with your model

Those are just some ideas. Deep learning models are really only as good as their training data so you probably should look more into that. Here's another link you can check out to see how to work with small datasets like yours. Another useful approach would be using grad-cam. This allows to see what your model considers from your input image to make its prediction so you can understand it better.

Hope this helps.