Explanation-based approach to incorporating domain knowledge into support vector machine: theory and applications

This is a comprehensive study of how to use domain knowledge to introduce domain-specific bias into learning systems. In this work, a Explanation Based Learning (EBL) approach is applied to mediate between the evidence in prior knowledge and the evidence in the training examples. Conventional EBL (DeJong, 1997) uses domain knowledge to explain the training examples, and generalize the explanations to obtain some deeper patterns, which, if believed, commits the learner to assigning classification labels to many unseen examples. In this study, we introduce a new learning framework, where those patterns obtained by EBL are used to introduce further inductive bias into a learning system. In this framework, EBL can be viewed as a mechanism to transform high-level domain-specific knowledge into special solution knowledge, which can then be used to introduce inductive bias into learning systems. We implemented our proposed explanation-based learning framework with three different approaches: phantom example approach, feature kernel approach and explanation-augmented SVM approach. In these approaches, we choose Support Vector Machines (SVM) as the inductive learner to demonstrate how domain knowledge can improve the performance of a learning system. We present both theoretical and empirical results to show that our approaches use domain knowledge to improve SVM's performance. The most novel aspect of our work is that EBL procedure encourages interactions between prior knowledge and the training examples. This allows our techniques to utilized information in the domain knowledge, which is otherwise difficult to incorporate into SVMs. Moreover, the inductive bias introduced into SVMs is calibrated for the given examples distribution, which potentially makes our approach more robust. We also present the comparison of the three proposed approaches, discuss about the related work, and point out some future work. We believe this work provide a first step towards a new research area in machine learning.