Multiview Latent Structure Learning: Local structure-guided cross-view discriminant analysis

Multiview subspace learning (MvSL) has attracted extensive research interest in many real-world applications, e.g., pattern recognition and computer vision. Existing MvSL approaches seek a discriminant common subspace by simultaneously maximizing the arithmetic mean of the between-class covariance and minimizing the global average of the within-class covariance. However, the arithmetic mean normally assigns equal weight to all between-class distances. Then, classes with small distances overlap with each other, and the largest between-class distance dominates the common subspace. In addition, the global average of within-class covariance (which is computed by the sum of within-class distances for all classes) significantly differs from the within-class covariance of a single class. This will inevitably cause the internal details of each class to be ignored and lead to lower recognition accuracy. To circumvent these issues, we develop a productive strategy to take more consideration of the relationship between any two classes and propose a novel Multiview Latent Structure Learning (MvLSL) model, which incorporates the weighted harmonic mean of pairwise between-class scatter and pairwise within-class scatter. Due to the adjustable dimensions of the common subspace, MvLSL can be naturally extended for dimensionality reduction (DR), and we utilize a reconstruction term to suppress data degradation and improve the DR performance. Our models are tested on six benchmark datasets with five evaluation criteria and compared with existing state-of-the-art methods, indicating our models’ effectiveness and superiority.