mRelief: A Reward Penalty Based Feature Subset Selection Considering Data Overlapping Problem.

Feature selection plays a vital role in machine learning and data mining by eliminating noisy and irrelevant attributes without compromising the classification performance. To select the best subset of features, we need to consider several issues such as the relationship among the features (interaction) and their relationship with the classes. Even though the state-of-the-art, Relief based feature selection methods can handle feature interactions, they often fail to capture the relationship of features with different classes. That is, a feature that can provide a clear boundary between two classes with a small average distance may be mistakenly ranked low compared to a feature that has a higher average distance with no clear boundary (data overlapping). Moreover, most of the existing methods provide a ranking of the given features rather than selecting a proper subset of the features. To address these issues, we propose a feature subset selection method namely modified Relief (mRelief) that can handle both feature interactions and data overlapping problems. Experimental results over twenty-seven benchmark datasets taken from different application areas demonstrate the superiority of mRelief over the state-of-the-art methods in terms of accuracies, number of the selected features, and the ability to identify the features (gene) to characterize a class (disease).