Post pruning is start from downward discarding subtree and include leaf node performance. so what is the best point or condition of the tree where we have to stop further pruning.
There are a variety of conditions we can use when deciding whether to prune a sub-tree or not after generating a decision tree model. There are three common approaches.
- We can prune branches with less support than a specific threshold. These are branches which were constructed using very few points from the training data.
- We can prune branches where the information gain from a split (or any other splitting measure we are using, like GINI), is smaller than a threshold.
- You can do what is done in Quinlan's 4.5 & C5.0 learners (which are the standard approaches; J48 is another implementation of the same algorithm). Quinlan performs a Chi-squared-like test for the relationship between the attribute we split upon and the target attribute. If the relationship is statistically significant, then the split is preserved. If not, it is not. The "confidence factor" parameter found in most implementations of these algorithms corresponds to the $\alpha$ value used in determining whether the relationship is considered significant. This approach captures the idea that we should prefer to keep branches that have few datapoints, but a very strong signal, or that have a weak signal, but very many datapoints supporting the pattern, since both cases are less likely to be overfitting that cases where we have weak signals and small numbers of points.