keras model accuracy not improving

Question

I am trying to do multi class(16) classification, however no matter what parameters or number of layers I use my accuracy is not improving, its in 30s the max I got was 43.

I have tried early stopping to red overfilling but my testing accuracy is still low.

I have 750 images in training and 350 in testing. I am also getting high traning accuracy vs low validation accuracy.

features_train=features_train/255
features_test=features_test/255

cnn = models.Sequential([
    layers.Conv2D(filters=16, kernel_size=(3, 3), activation='relu',  strides=(2, 2), padding="same", input_shape=(224, 224, 3)),

    layers.MaxPooling2D((2,2)),
    layers.Dropout(0.25),
    layers.Conv2D(32 ,(3, 3), activation='relu'),
    layers.MaxPooling2D((2,2)),
    layers.Dropout(0.25),
    layers.Conv2D(64, (3, 3), activation='relu'),
    layers.MaxPooling2D((2,2)),
    layers.Dropout(0.25),
    layers.Conv2D(128, (3, 3), activation='relu'),
 
    layers.MaxPooling2D((2,2)),
    
    
    layers.Flatten(),
    layers.Dense(64,activation='relu'),
    layers.Dense(16, activation='softmax')
])
cnn.compile(optimizer='adam',
             loss='categorical_crossentropy',
             metrics=['accuracy'])
cnn.fit(features_train,labels_train,epochs=20, batch_size = 4 ,validation_split = 0.25)

Answer 1

Based on having 16 classes, the naïve accuracy of a dummy classifier would be around 1/16 (6.25%) assuming class balance and the dummy classifier predicts each class according to uniform probabilities (see scikit-learn article on dummy classifiers). Therefore, your model is likely learning something useful beyond baseline.

Depending on the complexity of your images, your training set having only 750 images may be on the low side and may be the possible/likely culprit for low accuracy. Computer vision applications often require thousands of images for training. For example, see CIFAR-100, CIFAR-10, MNIST digits, and MNIST fashion datasets, which have thousands of images. For these large datasets, the achievable accuracy decreases as the complexity of the images increases (compare MNIST digits vs. MNIST fashion) and as the number of classes increases (compare CIFAR-100 vs. CIFAR-10).

Here is what I suggest for diagnosis:

1. Plot training and validation loss and accuracy vs. epoch to assess whether your model is overfitting / underfitting.

2. Determine the accuracy achievable by humans on the same dataset to assess what type of accuracy may be reasonable. Of course, AI can outperform or underperform humans, so take this with a grain of salt.

3. If this a public dataset, search the literature for what accuracy other investigators have achieved.

Here is what I suggest to boost accuracy after you have run the diagnostics:

1. Consider generating more real data and/or using Data Augmentation.

2. Tune hyperparameters based on whether you are overfitting or underfitting.

3. Consider using a pre-trained model.