Feature Decoupled of Deep Mutual Information Maximization

In deep learning, supervised learning techniques usually require a large amount of expensive labeled data to train the network, and the feature representations extracted by the model usually mix multiple attributes, resulting in feature representations that are difficult to decouple and are non-interpretable, which restricts the application and development of deep learning techniques, and for this reason, it is particularly important to study decoupled feature representation methods for unsupervised learning. Although the Learning deep representations by mutual information estimation and maximization (DIM) method achieves excellent results in unsupervised learning, the feature representations learned by the DIM method still suffer from the problem of difficult decoupling. decoupling problem. To address this problem, we minimize the mutual information between the intermediate layer feature representations learned by the hidden layer of the encoder during the encoder training process, so that the features learned by each filter are as uncorrelated as possible, thus realizing feature decoupling, and our method is called FP-DIM. Finally, the effectiveness of the FP-DIM method is verified on the CIFAR-10, STL-10, and Fashion-MNIST datasets. The experiments show that our proposed FP-DIM method is more significant for the learned decouplable middle layer feature representation. Finally, we also propose a reflection of future research for the FP-DIM method, aiming to provide a research idea and direction for solving unsupervised interpretable machine learning and to lay a solid theoretical and application foundation for machine learning fields such as image classification and migration learning.