Combining and Merging Deep Neural Networks for Arabic Text Categorization

Arabic text categorization is a challenging assignment due to the Arabic language's complexity and richness. Deep neural networks have shown promising performance in different text mining applications such as Information Retrieval, Text summarization, Document Clustering, etc. In this paper, we propose an efficient method for Arabic text categorization by combining and merging deep neural networks. First, we integrate the Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) on the top of Word Embedding representations to build the combined CNN-LSTM and LSTM-CNN models. Afterward, we align the output layers of the combined models and merge them into a unified model to generate an enriched text representation. Finally, these resultant representation vectors are passed to a fully connected layer then to an output one to predict texts categories. The proposed method effectively produces a compact model, taking advantages from the local feature extraction, contextual information, and implicit semantics within texts. We also benefit from the two networks' potentials to improve text categorization. To evaluate our method, we carry out several experiments on the OSAC dataset. Experimental results demonstrate the effectiveness of the proposed method compared to state-of-the-art.