I have a task for which I have to do image segmentation (cancer detection on MRIs). If possible, I would also like to include clinical data (i.e. numeric/categorical data which comes in the form of a table with features such as age, gender, ...).
I know that for classification purposes, it's possible to create a model that uses both numeric data as well as image data (as mentioned in the paper by Huang et al. : "Multimodal fusion with deep neural networks for leveraging CT imaging and electronic health record: a case‑study in pulmonary embolism detection"
The problem I have is that, for image segmentation tasks, it doesn't really make sense to me as to how to use both types of data.
In the above-mentioned paper, they create one model with only the image data and another with only the numeric data, and then they fuse them (there are multiple strategies for fusing them together). For classification tasks, it makes sense. However, for my task, it does not make sense to have a model which only uses the clinical data for image segmentation and that's where I get confused.
How do you think I should proceed with my task? Is it even possible to mix both types of data for image segmentation tasks?