Ant colony optimization with fuzzy-based ensemble of heuristics for ensemble feature selection

One of the crucial stages in machine learning in high-dimensional datasets is feature selection. Unrelated features weaknesses the efficiency of the model. However, merging several feature selection strategies is routine to solve this problem, the way to integrate feature selection methods is problematic. This paper presents a new ensemble of heuristics through fuzzy Type-I based on Ant Colony Optimization (ACO) for ensemble feature selection named Ant-EHFS. At first, three feature selection methods are run; then, the Euclidean Distance between each pair of features is computed as a heuristic (an M×M matrix is constructed), that M is the total of features. After that, a Type-I fuzzy is used individually to address various feature selections' uncertainty and estimate trustworthiness for each feature, as another heuristic. A complete weighted graph based on combining the two heuristics is then built; finally, ACO is applied to the complete graph for finding features that have the highest relevance together in the features space, which in each ant considers the reliability rate and Euclidean Distance of the destination node together for moving between nodes of the graph. Five and eight robust and well-known ensemble feature selection methods and primary feature selection methods, respectively, have been compared with Ant-EHFS on six high-dimensional datasets to show the proposed method's performance. The results have shown that the proposed method outperforms five ensemble feature selection methods and eight primary feature selections in Accuracy, Precision, Recall, and F1-score metrics.