I have a couple different segmentation tasks that I would like to perform on medical imaging data using CNN's. I'm currently trying to wrap my head around how well a 3D network might work, using a U-Net architecture, but I have some hesitation.
My specific questions are as follows:
Question 1 - Say we have medical imaging taken at different slices (different heights/depths) of the patient's body. In order for a 3D CNN to work properly on such data, do the images need to be taken at a heights that are close together, i.e. does the 3rd dimension need to be fairly continuous? For example, if you had a 3D stack/volume of 5 images that you wanted to feed to a 3D CNN like so,
- img_1_depth_0cm.png
- img_2_depth_5cm.png
- img_3_depth_10cm.png
- img_4_depth_15cm.png
- img_5_depth_20cm.png
and the images were taken 5 cm apart from one and other, I would imagine that a 3D convolution might not perform very well because of the 5 cm of depth in between images? Is this an incorrect assumption?
(As a reference point, this nice repository on GitHub was designed for training on images of the brain, but the images do appear to be fairly continuous/close together, almost like a video: https://github.com/ellisdg/3DUnetCNN)
Question 2 - For a 3D network to properly segment non-labelled volumes after training is finished, I know that the input data dimensions would have to match those of the training data.
But I would also think that the new images must have been taken in a similar fashion (taken at similar depths and in general a similar orientation to the training data) in order for the NN to be able to perform its task. So unless the medical imaging processes are always performed similarly across machines and hospitals, I'm guessing that the NN performance might vary pretty wildly when it tries to segment new data. Is this correct?
