I am developing a CNN model to recognize 24 hand-signs of American Sign Language. I have 2500 Images/hand-sign. The data split is:
Training = 1250 Images/hand-sign
Validation = 625 Images/hand-sign
Testing = 625 Images/hand-sign
How should I proceed with training the model?:
1. Should I develop a model starting from fewer hand-signs (like 5) and then increase them gradually?
2. Should I start models from scratch or use transfer learning (VGG16 or other)
Applying data augmentation, I did some tests with VGG16 and added a dense classifier at the end and received these accuracies:
Train: 0.87610877
Validation: 0.8867307
Test: 0.96533334
Test parameters:
NUM_CLASSES = 5
EPOCHS = 50
STEPS_PER_EPOCH = 125
VALIDATION_STEPS = 75
TEST_STEPS = 75
Framework = Keras, Tensorflow
OPTIMIZER = adam
Model:
model = Sequential([
Conv2D(32, (3, 3), activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),
MaxPooling2D(pool_size=(2,2)),
Conv2D(64, (3, 3), activation='relu'),
MaxPooling2D(pool_size=(2,2)),
Conv2D(128, (3, 3), activation='relu'),
MaxPooling2D(pool_size=(2,2)),
Conv2D(256, (3, 3), activation='relu'),
MaxPooling2D(pool_size=(2,2)),
Conv2D(512, (3, 3), activation='relu'),
MaxPooling2D(pool_size=(2,2)),
Flatten(),
Dense(512, activation='relu'),
Dense(NUM_CLASSES, activation='softmax')
])
If I try images with slightly different background and predict the classes (predict_classes()), I do not get accurate results. Any suggestions on how to make the model robust?