BACKGROUND: To apply multimodal machine learning (ML), the various data modalities typically come from the same example (e.g.,
chest X-ray (modality 1) and
cancer biomarkers (modality 2) come from the same
patient (the example)). The problem is that we often don't have that in public datasets. Instead, datasets are more commonly independent (e.g., a dataset of
chest X-rays from one set of patients and a dataset of
cancer biomarkers from a second set of patients).
QUESTION: Is there any validity to artificially creating "co-registered" datasets from such independent datasets for the ultimate purpose of leveraging multimodal ML (e.g., generate all possible pairs of
chest X-rays and
cancer biomarkers from different patients with cancer and generate the same for different patients without cancer)?
NOTE: It is understood that this approach is non-canonical and has flaws, but that is not the question here. I am more interested in learning whether this could be a second best option for researchers who lack co-registered samples but still want to develop multimodal ML models. Please provide your response along with some justification as to why this would or would not be valid "second best option".
New NOTE: I thought it was implied in the question post that I am aware that the interaction between modalities is not available to be leveraged by multimodal ML since this is really the crux of the problem with fusing independent datasets. However, the question still stands: "Would this be a valid second best option?"