The decision tree algorithm follows a recursive partitioning approach and operates greedily. At each step, it selects the best split point and the most suitable variable. The process continues iteratively until a stopping criterion is met.

The main steps involved in the algorithm are as follows:

1. Calculate the best split for each variable.
2. Select the variable with the best split.
3. Partition the data based on the selected split.
4. Repeat steps 1, 2, and 3 for each newly created partition until a stopping criterion is met.

Stopping Criteria in Decision Trees

• Tree Height Reaches a Predefined Limit

  Once the tree height reaches the value specified by the max_depth hyperparameter, the tree construction stops.




• Insignificant Information Gain After Partitioning

  One of the simplest stopping criteria is when a newly created partition consists primarily of data points from a single class. In such cases, further splitting does not provide significant information gain, making additional partitioning unnecessary.

Other important hyperparameters can also be used as stopping criteria, which are discussed here.