... and how do I reword my question in the title?
I have a dataset where each "instance" has a "series" of multiple photos taken from different angles. I need to classify each instance as a 0 or a 1.
A little over half of the images in each series probably do not contain the information required for a classification. Only some of the images are taken from an angle where the relevant clue is visible.
For training I have many such series and they are labelled at a series level, but not at an image level.
My current approach is to use a standard architecture like ResNet. I pass each image through the CNN then I combine the features by averaging, then put that through a sigmoid activated layer. I'm concerned that the network won't be able to learn because the "clue" is so buried among everything else.
Questions:
Is there a better/standard way to do this? Would going RNN help? What if the images are not really in a meaningful sequence?
If my way is good, is arithmetic averaging the right way to combine the features?
Would it be worth spending the time to label each image as "has positive clue"/"does not have positive clue"? Should I add a "not possible to tell"? What if it is possible to tell but it's just humans that can't tell?
