Questions tagged [gradient-boosting]
For questions related to gradient boosting, which is a machine learning technique that can be used for regression and classification problems and which produces a prediction model in the form of an ensemble of other smaller prediction models (typically decision trees).
13
questions
0
votes
0
answers
20
views
Multi-Variate Time-Series forecasting with XGBoost
I have trained an XGBoost model on a time-series dataset for predicting a value. The time series has 5 features and one label (the target value). The trained model works fine on both training and ...
- 181
0
votes
0
answers
6
views
Expected Revenue Using Gradient Boost for Regression
I have trained a ML algorithm (gradient boost) to do regression on banana prices, such that I can guess the selling price of any given banana. Using scikit's regression boost algorithm, I am able to ...
- 37
0
votes
0
answers
9
views
Tree boosting additive loss
In the XGBoost documentation, they specify that the additive training is done given an objective $obj^{(t)}$ defined as
$obj^{(t)} = \sum\limits_{i=1}^n \ell(y_i, \hat{y}_i^{(t-1)}+f_t(x_i)) + \sum\...
- 227
2
votes
1
answer
134
views
Why is the exponential loss used in this case?
I am reading the paper Tracking-by-Segmentation With Online Gradient Boosting Decision Tree. In Section 2.1, the paper says
Given training examples, $\left\{\left(\mathbf{x}_{i}, y_{i}\right) \mid \...
- 23
2
votes
1
answer
44
views
House price inflation modelling
I have a data set of house prices and their corresponding features (rooms, meter squared, etc). An additional feature is the sold date of the house. The aim is to create a model that can estimate the ...
- 35
1
vote
0
answers
21
views
React on train-validation curve after trening
I have a regression task that I tray to solve with AI.
I have around 6M rows with about 30 columns. (originally there was 100, but I reduce it with drop feature importance)
I understand basic ...
- 169
3
votes
0
answers
127
views
Can XGBoost solve XOR problem?
I've read that decision trees are able to solve XOR operation so I conclude that XGBoost algorithm can solve it as well.
But my tests on the datasets (datasets that should be highly "xor-ish"...
- 1,195
1
vote
1
answer
81
views
Has "deep vs. wide" been resolved?
All else being equal, including total neuron count, I give the following definitions:
wide is a parallel ensemble, where good chunks of the neurons have the same inputs because the inputs are shared ...
- 251
3
votes
0
answers
61
views
When do the ensemble methods beat neural networks?
In many applications and domains, computer vision, natural language processing, image segmentation, and many other tasks, neural networks (with a certain architecture) are considered to be by far the ...
- 1,894
5
votes
3
answers
639
views
How do weak learners become strong in boosting?
Boosting refers to a family of algorithms which converts weak learners to strong learners. How does it happen?
- 93
2
votes
1
answer
77
views
What are some applications where tree models perform better than neural networks?
Neural networks are known to be generally better modeling techniques as compared to tree-based models (such as decision trees). Are there any exceptions to this?
- 39
1
vote
0
answers
30
views
How would the "best function" been constructed if there are no computationally limitations?
I am reading the Wikipedia article on gradient boosting. There is written:
Unfortunately, choosing the best function $h$ at each step for an arbitrary loss function $L$ is a computationally ...
- 579
1
vote
0
answers
71
views
How can I use gradient boosting with multiple features?
I'm trying to use gradient boosting and I'm using sklearn's GradientBoostingClassifier class.
My problem is that I'm having a data frame with 5 columns and I want ...
- 111