Im trying to use Deep-Learning to recognize breast cancer on Mammography Images. But in the dataset every patient has a different (1-4) number of images taken. How can i deal with that? Generally i know that for varying input sizes (e.g. in NLP) you usually use RNN. But do you think it makes sense in this domain as well? Or are there other commonly used techniques for this problem?

Because in NLP RNN makes more sense (i think) because you have an order of the input word-vectors. But the Images are not ordered, they are 1-4 different features.

Thanks for your help!


1 Answer 1


There are are couple methods that you may wish to consider for this challenging but interesting problem.

In some cases, multiple images are captured because of artifacts in the initial images, so start by eliminating those. If there are post surgical mammograms, you should probably eliminate those too (but you decide as the domain expert).

One option is to develop a convolutional neural network that takes 4 images as inputs. Start by creating a dataset with a complete set of 4 images for each patient. If you only have 1 image, then copy that image 3 times. If you have two images, then duplicate each image to make 4. If you have 3 images, then randomly duplicate one of them. Alternatively, in the latter case, create 3 "new patients" with a different duplicated image.

A second option is to train a convolutional neural network that takes only one image as input. At time of inference, if your patient has multiple images, then apply the model to each image. Then aggregate the results to get a final classification for cancer status. This can be averaging the probabilities or max voting, for example.

Not sure about using recurrent neural networks for this problem. If you are looking for equivocal lesions on an initial mammogram that progressed to more definitive lesions on subsequent mammograms, then this may be an option. The presence (and growth) of a lesion in an earlier mammogram at the same anatomical region is certainly an important consideration for radiologists. A lot of this depends on what your objective is and how close in time the mammograms are (e.g., annual mammograms, repeat/follow-up mammograms).

The clinical considerations should inform the best machine learning strategy. As your images are not ordered (?and not dated), the ability to leverage clinical knowledge may be limited.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .