Arabic News Summarization based on T5 Transformer Approach

The problem of automatic text summarization is one of the main challenging problems in the field of natural language processing. With the huge amount of data available on the internet, automatic text summarization techniques become necessary to summarize this large number of documents to extract information quickly and efficiently. In this work, the automatic text summarization problem has been investigated using Transfer Learning with a customized Unified Text-to-Text Transformer T5 (t5-arabic-base) model. The t5-arabic-base was fine-tuned on Aljazeera.net news dataset and produced state-of-the-art performance in abstractive automatic text summarization. The experiments achieved F1-measure for ROUGE1, ROUGE2, and ROUGEL equal to 62.84%, 54.84%, 61.98%, respectively. Finally, we explained the model's reasoning process using heat maps and saliency maps. In addition to that, the model's sensitivity to slight perturbations to the input was discussed by using adversarial examples generated using Input Reduction and HotFlip techniques.