Fast and efficient exception tolerant ensemble for limited training

PRISM, RISE, C4.5 and CN2 are popular classification algorithms for solving binary and multi-class classification problems. These are simple yet powerful algorithms for limited training whereas a few state-of-the-art classifiers like Random Forest, Xgboost etc. fail to perform well without extensive training. In case of limited training exceptions play a vital role. A exception handling strategy can boost the performances of these algorithms. The underlying strategies and most importantly the output formats of these algorithms are completely different from one another. PRISM produces modular rules but with no scope of handling exceptions. RISE, however, provides the most intelligible form of rules but again the exceptions are not taken care of. C4.5, on the other hand, yields decision trees which are neither quite comprehensible nor too easy to manipulate by both machines and humans. Moreover, a tree cannot be exception tolerant. CN2 induces modular rules by finding out best complexes that cover maximum instances, and collectively cover all. This paper proposes an ensemble which uses these algorithms individually as base classifiers and improves the performance by using several methodologies like transposing the outputs into a similar format like that of RISE-induced rules, bagging, appending exceptions to rule set along with a default rule, eliminating inefficient rules and employing a new combination method called “Clustering of rules according to their specificity”. The ensemble is named as ETEL (Exception Tolerant Ensemble Learner) and empirical study shows that ETEL outperforms some of the state-of-the-art ensembles precisely AdaBoost and Random Forest significantly.