My Visual Search Model is only achieving an accuracy of about 42% If anyone can give me advice to drastically improve this number I would greatly appreciate it. Below is my current flow of image augmentation, resnet-50 finetuning, and prediction

I'm currently in the process of trying to fine-tune resnet-50 on trading cards for visual search. I'm currently freezing all layers to retain the pretrained learned (features?) and replacing the top layer with the number of classes i'm training on.

In this case 263/~18000 classes that exist. I'm generating 30 augmentations for each of the original images of the 236 classes I'm fine-tunning with and then resizing all images to 299 x299.

I'm using cross-entropy as my loss function. The training and validation loss converges with very minimal improvement after about 100 epochs.

I'm then using the final layer of the model to extract 2048 dimensional vector representations of each of the 18000 ground truth classes/images.

I upsert these vector representations for each of my classes, 1 for each class, into a Pinecone db.

I then use the same fine-tunned feature extractor to generate vector representations for a set of test images, query pinecone and compare with the ground-truths using Euclidean distance to retrieve the most similar vector and class for that vector.



You must log in to answer this question.