Group image classification for whether containing unrelated images

Question

I'm kind of new to computer vision, and wondering whether this is any existing researches / solutions to following scenarios.

Suppose I have a dataset, each data point contains a few images (< 20 images), and a corresponding label. The label indicate whether those images contain any unrelated images.

For example,
if the images are {apple, apple, apple} then the label is false.
if the images are {apple, basketball, basketball} then the label is true.

I would like to build a model to classify whether those group of images contain such unrelated images.

Seems the idea appeared in my mind is to have a model such as following

1. image decoder (e.g efficient net) to decoder each image in the data point. Then each datapoint contains a few images after decoding
2. a self-attention layer for every image above after decoding to measure a similarity matrix for all images in this data point.
3. flatten above similarity matrix, and then connect to a fc layer into a single node as the prediction result.

Loss function would just be the binary cross entropy with the actual label.

I searched a bit, seems there isn't too much researches about such group classification, so not very sure whether above model make much sense.. Any idea would be appreciate, thanks!

Answer 1

This is called multiple instance learning, and its deep version is called deep multiple instance learning, there are many algorithms to train a model considering multiple instances at the same time.

Here is a survey paper Multiple Instance Learning: A Survey of Problem Characteristics and Applications