BACKGROUND: To apply multimodal machine learning (ML), the various data modalities typically come from the same example (e.g., chest X-ray (modality 1) and cancer biomarkers (modality 2) come from the same patient (the example)). The problem is that we often don't have that in public datasets. Instead, datasets are more commonly independent (e.g., a dataset of chest X-rays from one set of patients and a dataset of cancer biomarkers from a second set of patients).
QUESTION: Is there any validity to artificially creating "co-registered" datasets from such independent datasets for the ultimate purpose of leveraging multimodal ML (e.g., generate all possible pairs of chest X-rays and cancer biomarkers from different patients with cancer and generate the same for different patients without cancer)?
NOTE: It is understood that this approach is non-canonical and has flaws, but that is not the question here. I am more interested in learning whether this could be a second best option for researchers who lack co-registered samples but still want to develop multimodal ML models. Please provide your response along with some justification as to why this would or would not be valid "second best option".
11/26/2022
New NOTE: I thought it was implied in the question post that I am aware that the interaction between modalities is not available to be leveraged by multimodal ML since this is really the crux of the problem with fusing independent datasets. However, the question still stands: "Would this be a valid second best option?"
