An Improved Deep Neural Network Based on Combination of GRU and Auto Encoder for Sentiment Analysis

Abstract Sentiment analysis is a particularly common task for determining user thoughts and has been widely used in Natural Language Processing (NLP) applications. Gated Recurrent Unit (GRU) was already effectively integrated into the NLP process with comparatively excellent results. GRU networks outperform traditional recurrent neural networks in sequential learning tasks and solve gradient vanishing and explosion limitation of RNNs. In this paper, a novel approach as known Normalize Auto-Encoded GRU (NAE-GRU) was proposed, in order to reduce dimensionality of data through an Auto-Encoder and enhance the performance of the proposed approach by using batch normalization. Empirically, we demonstrate that the proposed model, with minor hyperparameters modification, and statistic vectors optimization, achieves outstanding sentiment classification performance on benchmark datasets. The developed NAE-GRU approach outperforms than other different traditional methods in terms of accuracy and convergence rate. The experimental results have showed that the developed approach accomplished excellent performance than existing approaches on four benchmark datasets included, Amazon review, Yelp review, IMDB and SSTb. The experimental results have showed that the developed approach is proficient to reduce the loss function, and capture long-term relationships with an effective design that achieved excellent results as compared state-of-the-art methods.