Post pruning is start from downward discarding subtree and include leaf node performance. so what is the best point or condition of the tree where we have to stop further pruning.

  • 1
    $\begingroup$ Hi. You should really add more context to your questions. For example, it's not automatically clear what this "post pruning" procedure specifically does. You could describe it in the question or link to a website that does it for you. Otherwise, people probably won't know what exactly you're talking about. $\endgroup$ – Philip Raeisghasem Apr 6 '19 at 7:22
  • $\begingroup$ Maybe this might help: saedsayad.com/decision_tree_overfitting.htm? $\endgroup$ – nbro Apr 6 '19 at 8:31
  • $\begingroup$ post pruning is one of a method to avoid overfitting. it is use to reduce the complexity of tree. $\endgroup$ – hina munir Apr 6 '19 at 11:49
  • $\begingroup$ @hinamunir How "post-pruning" different from "pre-pruning"? I had only heard of "pruning" in general. Your question is actually a useful question. Please, edit your post to include the definition of your terms, like post-pruning. $\endgroup$ – nbro Apr 9 '19 at 10:20

There are a variety of conditions we can use when deciding whether to prune a sub-tree or not after generating a decision tree model. There are three common approaches.

  1. We can prune branches with less support than a specific threshold. These are branches which were constructed using very few points from the training data.
  2. We can prune branches where the information gain from a split (or any other splitting measure we are using, like GINI), is smaller than a threshold.
  3. You can do what is done in Quinlan's 4.5 & C5.0 learners (which are the standard approaches; J48 is another implementation of the same algorithm). Quinlan performs a Chi-squared-like test for the relationship between the attribute we split upon and the target attribute. If the relationship is statistically significant, then the split is preserved. If not, it is not. The "confidence factor" parameter found in most implementations of these algorithms corresponds to the $\alpha$ value used in determining whether the relationship is considered significant. This approach captures the idea that we should prefer to keep branches that have few datapoints, but a very strong signal, or that have a weak signal, but very many datapoints supporting the pattern, since both cases are less likely to be overfitting that cases where we have weak signals and small numbers of points.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.