0
$\begingroup$

What ensemble methods are used in the state-of-the-art models?

When I surveyed the state-of-the-art methods of classification and detection, e.g. ImageNet, COCO, etc., I noticed that are few or even no references to the use of ensemble methods like bagging or boosting.

Is it a bad idea to use them?

However, I observed that many use ensemble in Kaggle competitions.

What makes it so different between the two groups of researchers?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

In my opinion, it is not because ensemble methods are not good, just the state-of-the-art and Kaggle competitions are two different fields.

Kaggle competitions can be understood as an industry project where the target (accuracy, distance value, etc) is the most important, and they can select some computationally expensive way such as ensemble methods to reach it.

The state-of-the-art models in other ways belong to the research area, where the most important is the contribution for science, you can not just combine a lot of models then call it is the research (and so unfair with some small researcher groups). If you want to contribute something depend on ensemble idea, it should be like this paper.

Improve this answer
$\endgroup$
3
  • $\begingroup$ I think it's possible to publish something that only shows that the combination of multiple models can achieve better performance than single models (maybe in some applied ML journal or something), although no example comes to my mind right now. In any case, you're right that research is different than Kaggle competitions. In research, ideas and novelty are fundamental, while the goal in Kaggle competitions is to achieve the highest performance (according to some metric). $\endgroup$
    – nbro
    Dec 24, 2020 at 15:26
  • $\begingroup$ Note, though, a lot of research (especially in DL) that attracts people and funds is the one that shows a model is better than another according to some performance measure (for example, AlphaFold). Moreover, novelty doesn't necessarily need to be radical, but it can just be a combination or improvement of other ideas: this is actually very common. $\endgroup$
    – nbro
    Dec 24, 2020 at 15:31
  • $\begingroup$ @nbro Agree, ResNext in image classification or YOLO in object detection is a good example of ideas combination $\endgroup$
    – CuCaRot
    Dec 25, 2020 at 14:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .