Why is the research on artificial intelligence at this stage all researching on a separate ability? For example, train the visual ability of the computer alone, train the speech recognition ability alone, and train the natural language understanding ability alone. Why not train these abilities together at the same time? Does anyone think this will lead to better AI?

  • 2
    $\begingroup$ The short answer is that it isn't. $\endgroup$
    – Andy
    Feb 24 at 14:13
  • 1
    $\begingroup$ As the current answer mentioned, Multimodal Learning seems to fit for this, but maybe Ensemble Learning could fit, depending on how it's set up. $\endgroup$ Feb 24 at 14:46
  • 1
    $\begingroup$ @NordineLotfi Ensemble learning is where (to first approximation) you train multiple systems on the same data and they "vote" on the output. Multimodal learning is the correct term for what huang is asking about. $\endgroup$
    – Ray
    Feb 24 at 21:20
  • $\begingroup$ I agree, which is why I said "maybe" since I wasn't sure if there was a better term than this one :) @Ray $\endgroup$ Feb 24 at 21:39

1 Answer 1


There is a large field of AI that indeed does this. It is called multi modal learning. It is a very active research area, especially in the last few years.

For more information see: https://en.m.wikipedia.org/wiki/Multimodal_learning

  • 4
    $\begingroup$ Text-to-image generators are one good recent example of multimodal ML. Also may be worth explaining why this is a more tricky thing to train - usually limited by availability of training data correctly setup for multimodal learning, whilst auto-regressive and semi-supervised ML on a single input modality can often construct training data from many publicly available sources. $\endgroup$ Feb 24 at 16:10
  • $\begingroup$ Thanks for your answer, the "multimodal learning" you mentioned is quite close to my question (subjectively). But I think the question is pretty open, so I'm not going to accept this answer immediately and see if there are more different perspectives. $\endgroup$
    – huang
    Feb 24 at 22:49
  • $\begingroup$ @huang please consider accepting this answer $\endgroup$ Mar 8 at 14:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .