Why is tree pruning useful in decision tree induction What i

Why is tree pruning useful in decision tree induction? What is a drawback of using a separate set of tuples to evaluate pruning? Given a decision tree, you have the option of (a) converting the decision tree to rules and then pruning the resulting rules, or (b) pruning the decision tree and then converting the pruned tree to rules. Which option would you choose and why?

Solution

Why is tree pruning useful in decision tree induction? What is a drawback of using a separate set of tuples to evaluate pruning?

Answer:

The decision tree built may overfit the training data. There could be too many branches,
some of which may reflect anomalies in the training data due to noise or outliers. Tree
pruning addresses this issue of overfitting the data by removing the least reliable
branches (using statistical measures). This generally results in a more compact and
reliable decision tree that is faster and more accurate in its classification of data.
The drawback of using a separate set of tuples to evaluate pruning is that it may not be
representative of the training tuples used to create the original decision tree. If the
separate set of tuples are skewed, then using them to evaluate the pruned tree would not
be a good indicator of the pruned tree’s classification accuracy. Furthermore, using a
separate set of tuples to evaluate pruning means there are less tuples to use for creation
and testing of the tree. While this is considered a drawback in machine learning, it may
not be so in data mining due to the availability of larger data sets.

Given a decision tree, you have the option of (a) converting the decision tree to rules and then pruning the resulting rules, or (b) pruning the decision tree and then converting the pruned tree to rules. Which option would you choose and why?

Answer:

If pruning a subtree, we would remove the subtree completely with method (b). However, with method (a), if pruning a rule, we may remove any precondition of it. The latter is less restrictive.