Knowledge-Based Learning through Feature Generation

Machine learning algorithms have difficulties to generalize over a small set of examples. Humans can perform such a task by exploiting vast amount of background knowledge they possess and applying it on the target task. One method for enhancing learning algorithms with external knowledge is through feature generation. Several feature generation schemes have been proposed for different types of external knowledge.In this work we propose a method for exploiting external knowledge represented in a standard dataset format. With the explosion of machine learning research, the set of such datasets increases rapidly. In our work, we introduce a new algorithm for generating features based on a collection of auxiliary datasets. We assume that, in addition to the training set, we have access to a set of additional datasets. Unlike the transfer learning setup, we do not assume that the auxiliary datasets represent learning tasks that are similar to our original one. The algorithm finds features that are common to the training set and the auxiliary datasets. Based on these features and on examples from the auxiliary datasets, it induces predictors for new features from the auxiliary datasets. The induced predictors are then added to the original training set as generated features.