I'm kind of new to computer vision, and wondering whether this is any existing researches / solutions to following scenarios.

Suppose I have a dataset, each data point contains a few images (< 20 images), and a corresponding label. The label indicate whether those images contain any unrelated images.

For example,
if the images are {apple, apple, apple} then the label is false.
if the images are {apple, basketball, basketball} then the label is true.

I would like to build a model to classify whether those group of images contain such unrelated images.

Seems the idea appeared in my mind is to have a model such as following

  1. image decoder (e.g efficient net) to decoder each image in the data point. Then each datapoint contains a few images after decoding
  2. a self-attention layer for every image above after decoding to measure a similarity matrix for all images in this data point.
  3. flatten above similarity matrix, and then connect to a fc layer into a single node as the prediction result.

Loss function would just be the binary cross entropy with the actual label.

I searched a bit, seems there isn't too much researches about such group classification, so not very sure whether above model make much sense.. Any idea would be appreciate, thanks!


1 Answer 1


This is called multiple instance learning, and its deep version is called deep multiple instance learning, there are many algorithms to train a model considering multiple instances at the same time.

Here is a survey paper Multiple Instance Learning: A Survey of Problem Characteristics and Applications

  • 1
    $\begingroup$ Thanks for pointing to the keyword! that's exactly what I'm looking for :) $\endgroup$
    – misakayu
    Nov 7, 2022 at 8:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .