2
$\begingroup$

For example, if I have a problem in which I try to predict if it is a nice day for jogging from a corpus of images, I might first convert the images to text descriptions (ex. raining in forrest, cloudy in city) and then use a binary classifier to determine if those descriptions are describing a nice day for jogging or not.

Would a system like this, which takes the output of one model and uses it as the input for another, be considered Ensemble Learning?

$\endgroup$

2 Answers 2

3
$\begingroup$

No. Ensemble Learning (EL) is a way to improve the model performance, which usually means to reduce bias (i.e., get a better model class) or reduce variance (i.e., get better at generalizing across samplings of the dataset.)

Moreover, in EL you need to ensure a "property" (I'll explain in a bit) among the learners (i.e., the models in the ensemble.), which also defines the kind of EL:

  • In boosting we need the learners to be correlated (this is the property I was referring to) to each other to effectively reduce the bias.
  • Conversely, in bagging we need the models to do independent errors otherwise we can't reduce variance by averaging them out.

In your case you're chaining two models, thus not combining/aggregating their predictions. When you chain one or more models you have to consider that each model made an error, say $e$, and that each time you chain the next model would receive an input (which is the output of the previous model) that already incurs an error $e$, therefore predicting an output associated to an even higher error, say $e'$ with $e'>e$. Usually it happens that the whole error, say $E=e+e'+\cdots$, behaves like the sum of individual errors which can even grow quite easily, making your own pipeline drift away quite easily.

A way to mitigate this issue, is to train the whole pipeline of models in an end-to-end fashion (i.e., you back-propagate the error through all the models), instead of training each model in isolation until the local optimal is reached.

$\endgroup$
2
$\begingroup$

Would a system like this, which takes the output of one model and uses it as the input for another, be considered Ensemble Learning?

Not usually. The main criteria to consider something as an ensemble learning setup is that there are multiple different learning units -maybe different algorithm - all working with the same inputs and target values. They are then aggregated in some way (taking a mean value, voting, maybe weighted) to get the ensemble's output.

So to take your example, if you ran multiple types of binary classifier to classify the text at the end of the pipeline, and combined their outputs using a majority vote rule, then that part of your complex model would be considered as an ensemble.

These terms for system architecture - pipeline, ensemble, stacking etc - are not necessarily formal though. They are for communication about engineering setup, more than mathematical definitions.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .