I'm kind of new to computer vision, and wondering whether this is any existing researches / solutions to following scenarios.
Suppose I have a dataset, each data point contains a few images (< 20 images), and a corresponding label. The label indicate whether those images contain any unrelated images.
For example,
if the images are {apple, apple, apple} then the label is false.
if the images are {apple, basketball, basketball} then the label is true.
I would like to build a model to classify whether those group of images contain such unrelated images.
Seems the idea appeared in my mind is to have a model such as following
- image decoder (e.g efficient net) to decoder each image in the data point. Then each datapoint contains a few images after decoding
- a self-attention layer for every image above after decoding to measure a similarity matrix for all images in this data point.
- flatten above similarity matrix, and then connect to a fc layer into a single node as the prediction result.
Loss function would just be the binary cross entropy with the actual label.
I searched a bit, seems there isn't too much researches about such group classification, so not very sure whether above model make much sense.. Any idea would be appreciate, thanks!