# Do we train a logistic regression model using a dataset that is 3 times bigger than the validation dataset?

Suppose we have a data set $$X$$ that is split as $$X_{\text{train}}$$, $$X_{\text{val}}$$ and $$X_{\text{test}}$$ and the outcome variable is binary. Let's say we train three different models (logistic regression, random forest, and a support vector machine) using $$X_{\text{train}}$$. We then get predictions for $$X_{\text{val}}$$ using each of the three models.

In stacking, is it correct to say that we train a logistic regression model on a data set of dimension $$|X_{\text{val}}| \times 3$$ with the predicted values and actual values of the validation set? This logistic regression model is then used to predict outcomes for data in $$X_{\text{test}}$$?

• By stacking, do you mean ensemble learning?
– nbro
Apr 28, 2019 at 12:25
• Do you mind if I edit the title of your question to be a bit more general so it is more clear the context of the post? Apr 28, 2019 at 13:59

The predictions of each of your initial models will become a feature to feed the meta learner. If you use $$n$$ initial models, then for each example you will feed the meta learner $$n$$ features, each feature being a prediction from one of the initial models.
Note that this doesn’t mean the size of your dataset increases. Instead, each member of $$X_{val}$$ will be represented by $$n$$ features, with the $$k$$th feature being equal to the initial prediction of the $$k$$th initial predictor.
Note: in a sense if you are caching the intermediate predictions of each initial predictor, you’ll end up with $$n$$ times as many data points as there are entries in $$X_{val}$$. But these are simply each going to be a feature of another data point, so the dataset hasn’t really increased in size.