Progressive Forest: An Early Stopping Criteria for Building Ensembles

Decision forests improve their predictive power based on the combination of various decision trees. The number of trees to be used to achieve the best possible accuracy is not preset and has to be determined by a trial and error process. In many classification problems more trees are used than necessary. This paper introduces a new method, called Progressive Forest, that progressively evaluates the addition of new decision trees into a decision forest to decide when adding more trees is not longer useful. This method was incorporated into the construction schemes of Proactive Forest and Random Forest with very encouraging results. It is experimentally shown that Progressive Forest reduces the number of trees while maintaining the accuracy of the classification. Progressive Forest can be incorporated into any scheme of construction of ensemble, which presents similar characteristics to Random Forest.