HSI-BERT: Hyperspectral Image Classification Using the Bidirectional Encoder Representation From Transformers

Deep learning methods have been widely used in hyperspectral image classification and have achieved state-of-the-art performance. Nonetheless, the existing deep learning methods are restricted by a limited receptive field, inflexibility, and difficult generalization problems in hyperspectral image classification. To solve these problems, we propose HSI-BERT, where BERT stands for bidirectional encoder representations from transformers and HSI stands for hyperspectral imagery. The proposed HSI-BERT has a global receptive field that captures the global dependence among pixels regardless of their spatial distance. HSI-BERT is very flexible and enables the flexible and dynamic input regions. Furthermore, HSI-BERT has good generalization ability because the jointly trained HSI-BERT can be generalized from regions with different shapes without retraining. HSI-BERT is primarily built on a multihead self-attention (MHSA) mechanism in an MHSA layer. Moreover, several attentions are learned by different heads, and each head of the MHSA layer encodes the semantic context-aware representation to obtain discriminative features. Because all head-encoded features are merged, the resulting features exhibit spatial–spectral information that is essential for accurate pixel-level classification. Quantitative and qualitative results demonstrate that HSI-BERT outperforms any other CNN-based model in terms of both classification accuracy and computational time and achieves state-of-the-art performance on three widely used hyperspectral image data sets.