An ensemble deep learning method as data fusion system for remote sensing multisensor classification

Because of the great achievements in designing remote sensing sensors, the extraction of useful information from multisource remote sensing data remains a challenging problem. Most of the recent research projects have applied single deep learning systems for data fusion and classification. The idea of using ensemble deep learning algorithms through a multisensor fusion system can improve the performance of data fusion tasks. In this research, however, a multi-sensor classification strategy, which is based on deep learning ensemble procedure and decision fusion framework, is investigated for the fusion of Light Detection and Ranging (LiDAR), Hyperspectral Images (HS), and very high-resolution Visible (Vis) images. This research proposes a basic classifier based on deep Convolutional Neural Network (CNN) in which the softmax layer is replaced by a Support Vector Machine (CNN-SVM). Then, a random feature selection is applied to generate two separate CNN-SVM ensemble systems, one for LiDAR and Vis and the other one for HS data. To overcome the similarity and overfitting between the deep features and the classifiers provided by two ensemble systems and to select the best subsets of the classifiers, two diversity measures select the most diverse combinations of the classifiers. Finally, a decision fusion method combines the obtained diverse classifiers from CNN ensembles. Results demonstrate that the proposed method achieves higher accuracy, and its performance outperforms some of the existing methods. The proposed ensemble CNN method improved single deep CNN, random forest, and Adaboost between 2% to 10% in terms of classification accuracy.