2
$\begingroup$

... and how do I reword my question in the title?

I have a dataset where each "instance" has a "series" of multiple photos taken from different angles. I need to classify each instance as a 0 or a 1.

A little over half of the images in each series probably do not contain the information required for a classification. Only some of the images are taken from an angle where the relevant clue is visible.

For training I have many such series and they are labelled at a series level, but not at an image level.

My current approach is to use a standard architecture like ResNet. I pass each image through the CNN then I combine the features by averaging, then put that through a sigmoid activated layer. I'm concerned that the network won't be able to learn because the "clue" is so buried among everything else.

Questions:

  • Is there a better/standard way to do this? Would going RNN help? What if the images are not really in a meaningful sequence?

  • If my way is good, is arithmetic averaging the right way to combine the features?

  • Would it be worth spending the time to label each image as "has positive clue"/"does not have positive clue"? Should I add a "not possible to tell"? What if it is possible to tell but it's just humans that can't tell?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.